-
Notifications
You must be signed in to change notification settings - Fork 0
/
search.xml
1008 lines (478 loc) · 745 KB
/
search.xml
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
<?xml version="1.0" encoding="utf-8"?>
<search>
<entry>
<title>Ansible-part12: block rescue and always</title>
<link href="/2020/06/21/Ansible-part12-block-rescure-and-always/"/>
<url>/2020/06/21/Ansible-part12-block-rescure-and-always/</url>
<content type="html"><![CDATA[<h4 id="Blocks"><a href="#Blocks" class="headerlink" title="Blocks"></a>Blocks</h4><p>Blocks allow for logical grouping of tasks and in play error handling.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">- name: test blocks resuce and always</span><br><span class="line"> hosts: web2</span><br><span class="line"> tasks:</span><br><span class="line"> - name: first command</span><br><span class="line"> debug: </span><br><span class="line"> msg: "first message success"</span><br><span class="line"> - name: second command</span><br><span class="line"> command: /bin/false</span><br><span class="line"> - name: third command</span><br><span class="line"> debug:</span><br><span class="line"> msg: "this is third command"</span><br></pre></td></tr></table></figure><p>There are three tasks in this playbook, but the second one is a wrong command, which will failed to be executed.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/block-resuce-always$ ansible-playbook -i hosts site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test blocks resuce and always] ******************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [first command] **********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "first message sucess"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">TASK [second command] *********************************************************************************************************************************************************************************************************************************</span><br><span class="line">fatal: [cube4200]: FAILED! => {"changed": true, "cmd": ["/bin/false"], "delta": "0:00:00.011635", "end": "2020-06-21 21:31:18.222247", "msg": "non-zero return code", "rc": 1, "start": "2020-06-21 21:31:18.210612", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">cube4200 : ok=2 changed=0 unreachable=0 failed=1 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>Since the second faild, so the third one will be skipped. I don’t want this happen, So I use block to solove this.</p><h4 id="Blocks-error-handling"><a href="#Blocks-error-handling" class="headerlink" title="Blocks error handling"></a>Blocks error handling</h4><p>Blocks also introduce the ability to handle errors in a way similar to exceptions in most programming languages. Blocks only deal with ‘failed’ status of a task. A bad task definition or an unreachable host are not ‘rescuable’ errors.</p><p>three key words: block rescure always are similar as try exception and final in python.<br>block section: will execute normally<br>rescue section: will execute only when block section has error<br>always section: will run no matter what the task status is.</p><p>I add block into playbook</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">- name: test blocks recue and always</span><br><span class="line"> hosts: web2</span><br><span class="line"> tasks:</span><br><span class="line"> - block:</span><br><span class="line"> - name: test block, this is first command</span><br><span class="line"> debug: </span><br><span class="line"> msg: "first message success"</span><br><span class="line"> - name: second command</span><br><span class="line"> command: /bin/false</span><br><span class="line"> rescue:</span><br><span class="line"> - name: third command, this is excuted only when block failed, otherwise will be skipped</span><br><span class="line"> debug:</span><br><span class="line"> msg: "some command in block failed, so this message be excuted"</span><br></pre></td></tr></table></figure><p>after running</p><a id="more"></a><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/block-resuce-always$ ansible-playbook -i hosts site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test blocks resuce and always] ******************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test block, this is first command] **************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "first message success"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">TASK [second command] *********************************************************************************************************************************************************************************************************************************</span><br><span class="line">fatal: [cube4200]: FAILED! => {"changed": true, "cmd": ["/bin/false"], "delta": "0:00:00.010445", "end": "2020-06-21 21:42:37.100618", "msg": "non-zero return code", "rc": 1, "start": "2020-06-21 21:42:37.090173", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []}</span><br><span class="line"></span><br><span class="line">TASK [third command, this is excuted only when block failed, otherwise will be skipped] ***************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "some command in block failed, so this message be excuted"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">cube4200 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=1 ignored=0</span><br></pre></td></tr></table></figure><p>eventhough the second command failed, the third command still be excuted since it is in the rescue section.</p><p>I am testing another situation</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - block:</span><br><span class="line"> - name: test block, this is first command</span><br><span class="line"> debug: </span><br><span class="line"> msg: "first message success"</span><br><span class="line"> - name: second command</span><br><span class="line"> command: /bin/date</span><br><span class="line"> rescue:</span><br><span class="line"> - name: third command, this is excuted only when block failed, otherwise will be skipped</span><br><span class="line"> debug:</span><br><span class="line"> msg: "some command in block failed, so this message be excuted"</span><br></pre></td></tr></table></figure><p>In this playbook, the second command is correct. what will happed to rescue section?</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/block-resuce-always$ ansible-playbook -i hosts site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test blocks resuce and always] ******************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test block, this is first command] **************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "first message success"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">TASK [second command] *********************************************************************************************************************************************************************************************************************************</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">cube4200 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>It shows the rescue section isn’t be excuted any more.</p><p>Let’s look at <code>always</code> scenario</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - block:</span><br><span class="line"> - name: test block, this is first command</span><br><span class="line"> debug: </span><br><span class="line"> msg: "first message success"</span><br><span class="line"> - name: second command</span><br><span class="line"> command: /bin/date</span><br><span class="line"> rescue:</span><br><span class="line"> - name: third command, this is excuted only when block failed, otherwise will be skipped</span><br><span class="line"> debug:</span><br><span class="line"> msg: "some command in block failed, so this message be excuted"</span><br><span class="line"> always:</span><br><span class="line"> - name: test always. whatever this task have to be executed.</span><br><span class="line"> debug:</span><br><span class="line"> msg: "this section definitely be executed!!"</span><br></pre></td></tr></table></figure><p>whatever happen, the always section will be executed, but rescue section isn’t excuted since the second command successed.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/block-resuce-always$ ansible-playbook -i hosts site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test blocks resuce and always] ******************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test block, this is first command] **************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "first message success"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">TASK [second command] *********************************************************************************************************************************************************************************************************************************</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test always. whatever this task have to be executed.] *******************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "this section definitely be executed!!"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">cube4200 : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>last scenario</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - block:</span><br><span class="line"> - name: test block, this is first command</span><br><span class="line"> debug: </span><br><span class="line"> msg: "first message success"</span><br><span class="line"> - name: second command</span><br><span class="line"> command: /bin/da</span><br><span class="line"> rescue:</span><br><span class="line"> - name: third command, this is excuted only when block failed, otherwise will be skipped</span><br><span class="line"> debug:</span><br><span class="line"> msg: "some command in block failed, so this message be excuted"</span><br><span class="line"> always:</span><br><span class="line"> - name: test always. whatever this task have to be executed.</span><br><span class="line"> debug:</span><br><span class="line"> msg: "this section definitely be executed!!"</span><br></pre></td></tr></table></figure><p>the second command failed, both rescue and always will be excuted.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/block-resuce-always$ ansible-playbook -i hosts site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test blocks resuce and always] ******************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test block, this is first command] **************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "first message success"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">TASK [second command] *********************************************************************************************************************************************************************************************************************************</span><br><span class="line">fatal: [cube4200]: FAILED! => {"changed": false, "cmd": "/bin/da", "msg": "[Errno 2] No such file or directory", "rc": 2}</span><br><span class="line"></span><br><span class="line">TASK [third command, this is excuted only when block failed, otherwise will be skipped] ***************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "some command in block failed, so this message be excuted"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">TASK [test always. whatever this task have to be executed.] *******************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "this section definitely be executed!!"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">cube4200 : ok=4 changed=0 unreachable=0 failed=0 skipped=0 rescued=1 ignored=0</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part11: cron module to power off remote, Wake-on-LAN module to power on remote</title>
<link href="/2020/06/19/Ansible-part11-cron-module-to-power-off-remote-wake-on-lab-module-to-power-on-remote/"/>
<url>/2020/06/19/Ansible-part11-cron-module-to-power-off-remote-wake-on-lab-module-to-power-on-remote/</url>
<content type="html"><![CDATA[<h4 id="power-off-using-cron"><a href="#power-off-using-cron" class="headerlink" title="power off using cron"></a>power off using cron</h4><p>I am going to power off one of my node using cron module.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">- name: cron to poweroff</span><br><span class="line"> hosts: web2</span><br><span class="line"> become: yes</span><br><span class="line"></span><br><span class="line"> tasks:</span><br><span class="line"> - name: power off</span><br><span class="line"> cron:</span><br><span class="line"> name: "shutdown"</span><br><span class="line"> user: root</span><br><span class="line"> minute: "52"</span><br><span class="line"> hour: "16"</span><br><span class="line"> job: "sync; /sbin/shutdown"</span><br></pre></td></tr></table></figure><p>after running playbook, the remote will be shutdown on 16:42 everyday.</p><h4 id="Wake-on-LAN"><a href="#Wake-on-LAN" class="headerlink" title="Wake-on-LAN"></a>Wake-on-LAN</h4><p>Compare to shutdown, power on remote is a little bit hard. Ansible has a Wake-on-LAN module can do it, but remote needs has Wake-on-LAN option is enabled.<br>Wake-on-LAN (WOL) is an Ethernet networking standard that allows a server to be turned on by a network message.</p><p>If you don’t see this option in your BIOS or UEFI, check the computer or motherboard’s manual to see if it supports Wake-on-LAN.</p><p>To verify if remote supports Wake-on-LAN, use <code>ethtool -s eth0 wol g</code></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[anna@cube4200 ~]$ sudo ethtool enp3s0 |grep Wake</span><br><span class="line">Supports Wake-on: pumbg</span><br><span class="line">Wake-on: g</span><br></pre></td></tr></table></figure><p>It shows remote node cube4200 supports Wake-on-LAN</p><p>Let’s write playbook</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">- name: power on</span><br><span class="line"> hosts: web2</span><br><span class="line"> gather_facts: no</span><br><span class="line"></span><br><span class="line"> tasks:</span><br><span class="line"> - name: Send a magic Wake-on-LAN packet to turn on individual systems via gateway</span><br><span class="line"> wakeonlan:</span><br><span class="line"> mac: '00:f0:d2:d0:a0:0e'</span><br><span class="line"> broadcast: 255.255.255.255</span><br><span class="line"> delegate_to: localhost</span><br><span class="line"> - wakeonlan:</span><br><span class="line"> mac: '00:f0:d2:d0:a0:0e'</span><br><span class="line"> port: 9</span><br><span class="line"> delegate_to: localhost</span><br></pre></td></tr></table></figure><p>after running</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:/tmp$ ansible-playbook wakeonlan.yml </span><br><span class="line"></span><br><span class="line">PLAY [power on] ********************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Send a magic Wake-on-LAN packet to 00:00:5E:00:53:66] ************************************************************</span><br><span class="line">changed: [cube4200 -> localhost]</span><br><span class="line"></span><br><span class="line">TASK [wakeonlan] *******************************************************************************************************</span><br><span class="line">changed: [cube4200 -> localhost]</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************</span><br><span class="line">cube4200 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>waitting few seconds, I can ssh to cube4200</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:/tmp$ ssh anna@cube4200</span><br><span class="line">Last login: Thu Jun 18 00:56:29 2020 from 192.168.0.31</span><br></pre></td></tr></table></figure><p>Notice: <code>gather_facts: no</code> must be configured. Because <code>gather_facts</code> default is yes, so when playbook is excuted, it will gather facts from remote, which is not power on yet, so ansible will output error message as following:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:/tmp$ ansible-playbook wakeonlan.yml </span><br><span class="line"></span><br><span class="line">PLAY [power on] ********************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] *************************************************************************************************</span><br><span class="line">fatal: [cube4200]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: System is going down.\n\nConnection closed by 192.168.0.30 port 22", "unreachable": true}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************</span><br><span class="line">cube4200 : ok=0 changed=0 unreachable=1 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible-part10: net tool module uri</title>
<link href="/2020/06/16/Ansible-part10-net-tool-module-uri/"/>
<url>/2020/06/16/Ansible-part10-net-tool-module-uri/</url>
<content type="html"><![CDATA[<p>uri is a net tool moudle, which interacts with HTTP and HTTPS web services and supports Digest, Basic and WSSE HTTP authentication mechanisms.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: Check that a page returns a status 200 </span><br><span class="line"> uri:</span><br><span class="line"> url: http://www.example.com</span><br><span class="line"> return_content: yes</span><br><span class="line"> register: this</span><br><span class="line"> - debug:</span><br><span class="line"> msg: "{{this}}"</span><br></pre></td></tr></table></figure><p>look at the output</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/net-tools-modules$ ansible-playbook site1.yml </span><br><span class="line"></span><br><span class="line">PLAY [uri] ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] ********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">TASK [Check that a page returns a status 200] *********************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">TASK [debug] ******************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "msg": {</span><br><span class="line"> "accept_ranges": "bytes", </span><br><span class="line"> "age": "478999", </span><br><span class="line"> "cache_control": "max-age=604800", </span><br><span class="line"> "changed": false, </span><br><span class="line"> "connection": "close", </span><br><span class="line"> "content": "<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset=\"utf-8\" />\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n <style type=\"text/css\">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>Example Domain</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n", </span><br><span class="line"> "content_length": "1256", </span><br><span class="line"> "content_type": "text/html; charset=UTF-8", </span><br><span class="line"> "cookies": {}, </span><br><span class="line"> "cookies_string": "", </span><br><span class="line"> "date": "Wed, 17 Jun 2020 02:39:51 GMT", </span><br><span class="line"> "elapsed": 0, </span><br><span class="line"> "etag": "\"3147526947\"", </span><br><span class="line"> "expires": "Wed, 24 Jun 2020 02:39:51 GMT", </span><br><span class="line"> "failed": false, </span><br><span class="line"> "last_modified": "Thu, 17 Oct 2019 07:18:26 GMT", </span><br><span class="line"> "msg": "OK (1256 bytes)", </span><br><span class="line"> "redirected": false, </span><br><span class="line"> "server": "ECS (sjc/4E44)", </span><br><span class="line"> "status": 200, </span><br><span class="line"> "url": "http://www.example.com", </span><br><span class="line"> "vary": "Accept-Encoding", </span><br><span class="line"> "x_cache": "HIT"</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": {</span><br><span class="line"> "accept_ranges": "bytes", </span><br><span class="line"> "age": "478426", </span><br><span class="line"> "cache_control": "max-age=604800", </span><br><span class="line"> "changed": false, </span><br><span class="line"> "connection": "close", </span><br><span class="line"> "content": "<!doctype html>\n<html>\n<head>\n <title>Example Domain</title>\n\n <meta charset=\"utf-8\" />\n <meta http-equiv=\"Content-type\" content=\"text/html; charset=utf-8\" />\n <meta name=\"viewport\" content=\"width=device-width, initial-scale=1\" />\n <style type=\"text/css\">\n body {\n background-color: #f0f0f2;\n margin: 0;\n padding: 0;\n font-family: -apple-system, system-ui, BlinkMacSystemFont, \"Segoe UI\", \"Open Sans\", \"Helvetica Neue\", Helvetica, Arial, sans-serif;\n \n }\n div {\n width: 600px;\n margin: 5em auto;\n padding: 2em;\n background-color: #fdfdff;\n border-radius: 0.5em;\n box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);\n }\n a:link, a:visited {\n color: #38488f;\n text-decoration: none;\n }\n @media (max-width: 700px) {\n div {\n margin: 0 auto;\n width: auto;\n }\n }\n </style> \n</head>\n\n<body>\n<div>\n <h1>Example Domain</h1>\n <p>This domain is for use in illustrative examples in documents. You may use this\n domain in literature without prior coordination or asking for permission.</p>\n <p><a href=\"https://www.iana.org/domains/example\">More information...</a></p>\n</div>\n</body>\n</html>\n", </span><br><span class="line"> "content_length": "1256", </span><br><span class="line"> "content_type": "text/html; charset=UTF-8", </span><br><span class="line"> "cookies": {}, </span><br><span class="line"> "cookies_string": "", </span><br><span class="line"> "date": "Wed, 17 Jun 2020 02:39:50 GMT", </span><br><span class="line"> "elapsed": 0, </span><br><span class="line"> "etag": "\"3147526947\"", </span><br><span class="line"> "expires": "Wed, 24 Jun 2020 02:39:50 GMT", </span><br><span class="line"> "failed": false, </span><br><span class="line"> "last_modified": "Thu, 17 Oct 2019 07:18:26 GMT", </span><br><span class="line"> "msg": "OK (1256 bytes)", </span><br><span class="line"> "redirected": false, </span><br><span class="line"> "server": "ECS (sjc/4E76)", </span><br><span class="line"> "status": 200, </span><br><span class="line"> "url": "http://www.example.com", </span><br><span class="line"> "vary": "Accept-Encoding", </span><br><span class="line"> "x_cache": "HIT"</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ansible-node1 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=3 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>Notice: if there is no <code>return_content: yes</code>, the output will only parts of above info.<br>the status is 200, which means it is normal.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part9: fetch file from remote</title>
<link href="/2020/06/16/Ansible-part9-fetch-url-module/"/>
<url>/2020/06/16/Ansible-part9-fetch-url-module/</url>
<content type="html"><![CDATA[<p>file module has a fetch module, which is the same as copy but in reverse.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: fetech</span><br><span class="line"> fetch: </span><br><span class="line"> src: /tmp/ansible-file-test.txt</span><br><span class="line"> dest: /tmp/</span><br><span class="line"> flat:</span><br></pre></td></tr></table></figure><p>go to my local machine to see /tmp directory</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/files-modules/inventory$ ll /tmp/ansible-node1/tmp/ansible-file-test.txt </span><br><span class="line">-rw-rw-r-- 1 anna anna 10 Jun 16 18:18 /tmp/ansible-node1/tmp/ansible-file-test.txt</span><br><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/files-modules/inventory$ ll /tmp/cube4200/tmp/ansible-file-test.txt </span><br><span class="line">-rw-rw-r-- 1 anna anna 0 Jun 16 18:18 /tmp/cube4200/tmp/ansible-file-test.txt</span><br></pre></td></tr></table></figure><p>there are two hosts directories which are relate to my two hosts.</p><p>for the <code>flat</code>, it means ansible will not distinct the files from different hosts, the latest file will overwrite the old one.<br>for example</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: fetech</span><br><span class="line"> fetch: </span><br><span class="line"> src: /tmp/ansible-file-test.txt</span><br><span class="line"> dest: /tmp/</span><br><span class="line"> flat: yes</span><br></pre></td></tr></table></figure><p>notice: the default of flat is no, I am turning on it here, it will let the new file overwrite old file</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/files-modules/inventory$ ll /tmp</span><br><span class="line">total 76</span><br><span class="line">drwxrwxrwt 18 root root 4096 Jun 16 18:28 ./</span><br><span class="line">drwxr-xr-x 24 root root 4096 Jun 9 20:21 ../</span><br><span class="line">-rw-rw-r-- 1 anna anna 0 Jun 16 18:28 ansible-file-test.txt</span><br></pre></td></tr></table></figure><p>from above, I can see there is only one <code>ansible-file-test.txt</code> file which is from cube4200, and there is no file from ansible-node1.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible-part8: Ansible lint</title>
<link href="/2020/06/13/Ansible-part8-Ansible-lint/"/>
<url>/2020/06/13/Ansible-part8-Ansible-lint/</url>
<content type="html"><![CDATA[<p>Ansible lint: Ansible Lint is a commandline tool for linting playbooks. Use it to detect behaviors and practices that could potentially be improved.</p><h4 id="Install"><a href="#Install" class="headerlink" title="Install"></a>Install</h4><p>At first I try to Use pip install ansible-int, it failed.<br>then I try the apt install.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ sudo apt install ansible-lint</span><br><span class="line">[sudo] password for anna: </span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">The following packages were automatically installed and are no longer required:</span><br><span class="line"> golang-1.10-go golang-1.10-race-detector-runtime golang-1.10-src golang-race-detector-runtime golang-src ieee-data pkg-config python-certifi python-chardet python-jmespath python-kerberos python-libcloud python-lockfile python-netaddr</span><br><span class="line"> python-openssl python-requests python-selinux python-simplejson python-urllib3 python-xmltodict</span><br><span class="line">Use 'sudo apt autoremove' to remove them.</span><br><span class="line">The following NEW packages will be installed:</span><br><span class="line"> ansible-lint</span><br><span class="line">0 upgraded, 1 newly installed, 0 to remove and 34 not upgraded.</span><br><span class="line">Need to get 24.8 kB of archives.</span><br><span class="line">After this operation, 146 kB of additional disk space will be used.</span><br><span class="line">Get:1 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 ansible-lint all 3.4.20+git.20180203-1 [24.8 kB]</span><br><span class="line">Fetched 24.8 kB in 0s (76.1 kB/s) </span><br><span class="line">Selecting previously unselected package ansible-lint.</span><br><span class="line">(Reading database ... 240199 files and directories currently installed.)</span><br><span class="line">Preparing to unpack .../ansible-lint_3.4.20+git.20180203-1_all.deb ...</span><br><span class="line">Unpacking ansible-lint (3.4.20+git.20180203-1) ...</span><br><span class="line">Setting up ansible-lint (3.4.20+git.20180203-1) ...</span><br><span class="line">Processing triggers for man-db (2.8.3-2ubuntu0.1) ...</span><br><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ ansible-lint --version</span><br><span class="line">ansible-lint 3.4.20</span><br></pre></td></tr></table></figure><p>It shows the lint version is 3.4.20</p><h4 id="use-lint-to-test-yml-file"><a href="#use-lint-to-test-yml-file" class="headerlink" title="use lint to test yml file"></a>use lint to test yml file</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line"></span><br><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/packaging_modules/inventory$ ansible-lint site.yml</span><br><span class="line">[ANSIBLE0002] Trailing whitespace</span><br><span class="line">site.yml:10</span><br><span class="line"> name: </span><br><span class="line"></span><br><span class="line">[ANSIBLE0010] Package installs should not use latest</span><br><span class="line">site.yml:16</span><br><span class="line">Task/Handler: test apt module</span><br></pre></td></tr></table></figure><p>then I go to line 10 to add ‘test package module’ after name, and save playbook.<br>after run ansible-lint</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/packaging_modules/inventory$ ansible-lint site.yml</span><br><span class="line">[ANSIBLE0010] Package installs should not use latest</span><br><span class="line">site.yml:16</span><br><span class="line">Task/Handler: test apt module</span><br></pre></td></tr></table></figure><p>It only shows the second code which needs improved. </p><p>after I commented out the <code>state: latest</code>, the output of <code>ansible-lint site.yml</code> shows nothing, which means nothing needs improve in this site.yml file.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible-part7: set facts, set variable</title>
<link href="/2020/06/13/Ansible-part7-set-facts-set-variable/"/>
<url>/2020/06/13/Ansible-part7-set-facts-set-variable/</url>
<content type="html"><![CDATA[<h4 id="what-is-set-fact"><a href="#what-is-set-fact" class="headerlink" title="what is set_fact"></a>what is set_fact</h4><p>Set fact is a module which allows setting new variables. These variables are set on a host-by-host base just like facts discovered by the setup module.</p><p>In ansible, there are some default info of hosts in <code>hostvars</code> variable, such as “ansible_check_mode”, “group_names” etc, and also the “ansible_facts”, which is not default show up, only when <code>gather_facts</code> is set to “yes”, it will show the details of ansible_fact. Except these two kinds of info, I can also use set_fact module to create my own info pair.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">- name: set fact</span><br><span class="line"> hosts: all</span><br><span class="line"> gather_facts: no</span><br><span class="line"></span><br><span class="line"> tasks:</span><br><span class="line"> - name: gather facts</span><br><span class="line"> debug:</span><br><span class="line"> var: hostvars[ansible_host] #ansible_host is variable name, which means cube4200 and ansible-node1 in this case</span><br></pre></td></tr></table></figure><p>Let’s look at he output of this playbook,</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [set fact] ************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [gather facts] ********************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "hostvars[ansible_host]": {</span><br><span class="line"> "ansible_check_mode": false, </span><br><span class="line"> "ansible_connection": "ssh", </span><br><span class="line"> "ansible_diff_mode": false, </span><br><span class="line"> "ansible_facts": {}, </span><br><span class="line"> "ansible_forks": 5, </span><br><span class="line"> "ansible_inventory_sources": [</span><br><span class="line"> "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_playbook_python": "/usr/bin/python2", </span><br><span class="line"> "ansible_run_tags": [</span><br><span class="line"> "all"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_skip_tags": [], </span><br><span class="line"> "ansible_user": "anna", </span><br><span class="line"> "ansible_verbosity": 0, </span><br><span class="line"> "ansible_version": {</span><br><span class="line"> "full": "2.9.9", </span><br><span class="line"> "major": 2, </span><br><span class="line"> "minor": 9, </span><br><span class="line"> "revision": 9, </span><br><span class="line"> "string": "2.9.9"</span><br><span class="line"> }, </span><br><span class="line"> "group_names": [</span><br><span class="line"> "web2"</span><br><span class="line"> ], </span><br><span class="line"> "groups": {</span><br><span class="line"> "all": [</span><br><span class="line"> "cube4200", </span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "ungrouped": [], </span><br><span class="line"> "web1": [</span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "web2": [</span><br><span class="line"> "cube4200"</span><br><span class="line"> ]</span><br><span class="line"> }, </span><br><span class="line"> "inventory_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module", </span><br><span class="line"> "inventory_file": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts", </span><br><span class="line"> "inventory_hostname": "cube4200", </span><br><span class="line"> "inventory_hostname_short": "cube4200", </span><br><span class="line"> "omit": "__omit_place_holder__a53469922527cea947bde7fea01f0f911fe92d8c", </span><br><span class="line"> "playbook_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module"</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "hostvars[ansible_host]": {</span><br><span class="line"> "ansible_check_mode": false, </span><br><span class="line"> "ansible_connection": "ssh", </span><br><span class="line"> "ansible_diff_mode": false, </span><br><span class="line"> "ansible_facts": {}, </span><br><span class="line"> "ansible_forks": 5, </span><br><span class="line"> "ansible_inventory_sources": [</span><br><span class="line"> "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_playbook_python": "/usr/bin/python2", </span><br><span class="line"> "ansible_run_tags": [</span><br><span class="line"> "all"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_skip_tags": [], </span><br><span class="line"> "ansible_user": "pi", </span><br><span class="line"> "ansible_verbosity": 0, </span><br><span class="line"> "ansible_version": {</span><br><span class="line"> "full": "2.9.9", </span><br><span class="line"> "major": 2, </span><br><span class="line"> "minor": 9, </span><br><span class="line"> "revision": 9, </span><br><span class="line"> "string": "2.9.9"</span><br><span class="line"> }, </span><br><span class="line"> "group_names": [</span><br><span class="line"> "web1"</span><br><span class="line"> ], </span><br><span class="line"> "groups": {</span><br><span class="line"> "all": [</span><br><span class="line"> "cube4200", </span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "ungrouped": [], </span><br><span class="line"> "web1": [</span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "web2": [</span><br><span class="line"> "cube4200"</span><br><span class="line"> ]</span><br><span class="line"> }, </span><br><span class="line"> "inventory_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module", </span><br><span class="line"> "inventory_file": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts", </span><br><span class="line"> "inventory_hostname": "ansible-node1", </span><br><span class="line"> "inventory_hostname_short": "ansible-node1", </span><br><span class="line"> "omit": "__omit_place_holder__a53469922527cea947bde7fea01f0f911fe92d8c", </span><br><span class="line"> "playbook_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module"</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *****************************************************************************</span><br><span class="line">ansible-node1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>eventhought <code>gather_facts</code> is “no”, ansible still show some basic info of every hosts.<br>If <code>gather_facts</code> is “yes”, except the default info, ansible will show <code>ansible-facts</code> info which included a lots of system info of every hosts.</p><a id="more"></a><p>If I add <code>set_fact</code> module, and create a variable named <code>test_set_fact</code>, it will show in the output of hostvars, notice line 56 and line 105</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">- name: set fact</span><br><span class="line"> hosts: all</span><br><span class="line"> gather_facts: no</span><br><span class="line"></span><br><span class="line"> tasks:</span><br><span class="line"> - name: test facts</span><br><span class="line"> set_fact:</span><br><span class="line"> test_set_fact: "variable from set fact"</span><br><span class="line"> - debug:</span><br><span class="line"> var: hostvars[ansible_host] #ansible_host is variable name, which means cube4200 and ansible-node1 in this case</span><br></pre></td></tr></table></figure><p>Let’s look at the output</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [set fact] ***************************************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [test facts] *************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">TASK [debug] ******************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "hostvars[ansible_host]": {</span><br><span class="line"> "ansible_check_mode": false, </span><br><span class="line"> "ansible_connection": "ssh", </span><br><span class="line"> "ansible_diff_mode": false, </span><br><span class="line"> "ansible_facts": {}, </span><br><span class="line"> "ansible_forks": 5, </span><br><span class="line"> "ansible_inventory_sources": [</span><br><span class="line"> "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_playbook_python": "/usr/bin/python2", </span><br><span class="line"> "ansible_run_tags": [</span><br><span class="line"> "all"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_skip_tags": [], </span><br><span class="line"> "ansible_user": "anna", </span><br><span class="line"> "ansible_verbosity": 0, </span><br><span class="line"> "ansible_version": {</span><br><span class="line"> "full": "2.9.9", </span><br><span class="line"> "major": 2, </span><br><span class="line"> "minor": 9, </span><br><span class="line"> "revision": 9, </span><br><span class="line"> "string": "2.9.9"</span><br><span class="line"> }, </span><br><span class="line"> "group_names": [</span><br><span class="line"> "web2"</span><br><span class="line"> ], </span><br><span class="line"> "groups": {</span><br><span class="line"> "all": [</span><br><span class="line"> "cube4200", </span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "ungrouped": [], </span><br><span class="line"> "web1": [</span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "web2": [</span><br><span class="line"> "cube4200"</span><br><span class="line"> ]</span><br><span class="line"> }, </span><br><span class="line"> "inventory_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module", </span><br><span class="line"> "inventory_file": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts", </span><br><span class="line"> "inventory_hostname": "cube4200", </span><br><span class="line"> "inventory_hostname_short": "cube4200", </span><br><span class="line"> "omit": "__omit_place_holder__d4f2446f1f2a785d53a7d7fe3b7b731b4fa322b3", </span><br><span class="line"> "playbook_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module", </span><br><span class="line"> "test_set_fact": "variable from set fact"</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "hostvars[ansible_host]": {</span><br><span class="line"> "ansible_check_mode": false, </span><br><span class="line"> "ansible_connection": "ssh", </span><br><span class="line"> "ansible_diff_mode": false, </span><br><span class="line"> "ansible_facts": {}, </span><br><span class="line"> "ansible_forks": 5, </span><br><span class="line"> "ansible_inventory_sources": [</span><br><span class="line"> "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_playbook_python": "/usr/bin/python2", </span><br><span class="line"> "ansible_run_tags": [</span><br><span class="line"> "all"</span><br><span class="line"> ], </span><br><span class="line"> "ansible_skip_tags": [], </span><br><span class="line"> "ansible_user": "pi", </span><br><span class="line"> "ansible_verbosity": 0, </span><br><span class="line"> "ansible_version": {</span><br><span class="line"> "full": "2.9.9", </span><br><span class="line"> "major": 2, </span><br><span class="line"> "minor": 9, </span><br><span class="line"> "revision": 9, </span><br><span class="line"> "string": "2.9.9"</span><br><span class="line"> }, </span><br><span class="line"> "group_names": [</span><br><span class="line"> "web1"</span><br><span class="line"> ], </span><br><span class="line"> "groups": {</span><br><span class="line"> "all": [</span><br><span class="line"> "cube4200", </span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "ungrouped": [], </span><br><span class="line"> "web1": [</span><br><span class="line"> "ansible-node1"</span><br><span class="line"> ], </span><br><span class="line"> "web2": [</span><br><span class="line"> "cube4200"</span><br><span class="line"> ]</span><br><span class="line"> }, </span><br><span class="line"> "inventory_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module", </span><br><span class="line"> "inventory_file": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module/hosts", </span><br><span class="line"> "inventory_hostname": "ansible-node1", </span><br><span class="line"> "inventory_hostname_short": "ansible-node1", </span><br><span class="line"> "omit": "__omit_place_holder__d4f2446f1f2a785d53a7d7fe3b7b731b4fa322b3", </span><br><span class="line"> "playbook_dir": "/home/anna/Desktop/ansible-code/inventory/modules/set-fact module", </span><br><span class="line"> "test_set_fact": "variable from set fact"</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ansible-node1 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=2 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>At last line of every hosts,I can see the <code>"test_set_fact": "variable from set fact"</code>, which is the variable and value I set in the playbook before.</p><h4 id="How-to-use-set-fact-varible"><a href="#How-to-use-set-fact-varible" class="headerlink" title="How to use set_fact varible"></a>How to use set_fact varible</h4><p>I can call this variable directly in playbook</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test facts</span><br><span class="line"> set_fact:</span><br><span class="line"> test_set_fact: "variable from set fact"</span><br><span class="line"> - debug:</span><br><span class="line"> var: test_set_fact</span><br></pre></td></tr></table></figure><p>debug module will show only test_set_fact variable</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [set fact] ***************************************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [test facts] *************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">TASK [debug] ******************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "test_set_fact": "variable from set fact"</span><br><span class="line">}</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "test_set_fact": "variable from set fact"</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>another example<br>I am going to get the every hosts’s hostname info.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test facts</span><br><span class="line"> set_fact:</span><br><span class="line"> test_set_fact: "variable from set fact"</span><br><span class="line"> - command: cat /etc/hosts</span><br><span class="line"> register: hosts_result</span><br><span class="line"> - debug:</span><br><span class="line"> var: hosts_result</span><br></pre></td></tr></table></figure><p>I put all the output of command <code>cat /etc/hosts</code> into variable named <code>hosts_result</code>, then it will show like this</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [set fact] ***************************************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [test facts] *************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">TASK [command] ****************************************************************************************************************************************************************************************************************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [debug] ******************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "hosts_result": {</span><br><span class="line"> "ansible_facts": {</span><br><span class="line"> "discovered_interpreter_python": "/usr/bin/python"</span><br><span class="line"> }, </span><br><span class="line"> "changed": true, </span><br><span class="line"> "cmd": [</span><br><span class="line"> "cat", </span><br><span class="line"> "/etc/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "delta": "0:00:00.011381", </span><br><span class="line"> "end": "2020-06-13 14:38:12.510727", </span><br><span class="line"> "failed": false, </span><br><span class="line"> "rc": 0, </span><br><span class="line"> "start": "2020-06-13 14:38:12.499346", </span><br><span class="line"> "stderr": "", </span><br><span class="line"> "stderr_lines": [], </span><br><span class="line"> "stdout_lines": [</span><br><span class="line"> "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4", </span><br><span class="line"> "::1 localhost localhost.localdomain localhost6 localhost6.localdomain6"</span><br><span class="line"> ]</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "hosts_result": {</span><br><span class="line"> "ansible_facts": {</span><br><span class="line"> "discovered_interpreter_python": "/usr/bin/python"</span><br><span class="line"> }, </span><br><span class="line"> "changed": true, </span><br><span class="line"> "cmd": [</span><br><span class="line"> "cat", </span><br><span class="line"> "/etc/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "delta": "0:00:00.014459", </span><br><span class="line"> "end": "2020-06-13 21:38:08.778027", </span><br><span class="line"> "failed": false, </span><br><span class="line"> "rc": 0, </span><br><span class="line"> "start": "2020-06-13 21:38:08.763568", </span><br><span class="line"> "stderr": "", </span><br><span class="line"> "stderr_lines": [], </span><br><span class="line"> "stdout_lines": [</span><br><span class="line"> "127.0.0.1\tlocalhost", </span><br><span class="line"> "::1\t\tlocalhost ip6-localhost ip6-loopback", </span><br><span class="line"> "ff02::1\t\tip6-allnodes", </span><br><span class="line"> "ff02::2\t\tip6-allrouters", </span><br><span class="line"> "", </span><br><span class="line"> "127.0.1.1\ttreehouses"</span><br><span class="line"> ], </span><br><span class="line"> "warnings": [</span><br><span class="line"> "Platform linux on host ansible-node1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information."</span><br><span class="line"> ]</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP ********************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ansible-node1 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>but I only want the “stdout_lines” info, so I am going to set “stdout_lines” to a variable named <code>host_ip</code>, and use set_fact module to do this, modified the playbook like this:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - command: cat /etc/hosts</span><br><span class="line"> register: hosts_result</span><br><span class="line"> - set_fact:</span><br><span class="line"> host_ip: "{{hosts_result['stdout_lines']}}"</span><br><span class="line"> - debug:</span><br><span class="line"> var: host_ip</span><br></pre></td></tr></table></figure><p>So the output will be like this</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [set fact] ************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [command] *************************************************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [set_fact] ************************************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">TASK [debug] ***************************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "host_ip": [</span><br><span class="line"> "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4", </span><br><span class="line"> "::1 localhost localhost.localdomain localhost6 localhost6.localdomain6"</span><br><span class="line"> ]</span><br><span class="line">}</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "host_ip": [</span><br><span class="line"> "127.0.0.1\tlocalhost", </span><br><span class="line"> "::1\t\tlocalhost ip6-localhost ip6-loopback", </span><br><span class="line"> "ff02::1\t\tip6-allnodes", </span><br><span class="line"> "ff02::2\t\tip6-allrouters", </span><br><span class="line"> "", </span><br><span class="line"> "127.0.1.1\ttreehouses"</span><br><span class="line"> ]</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *****************************************************************************</span><br><span class="line">ansible-node1 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=3 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line"></span><br><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/set-fact module$</span><br></pre></td></tr></table></figure><p>It will show only the host ip which I want.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part6: ansible vault encrypt file</title>
<link href="/2020/06/03/Ansible-part6-ansible-vault-encrypt-file/"/>
<url>/2020/06/03/Ansible-part6-ansible-vault-encrypt-file/</url>
<content type="html"><![CDATA[<p>Ansible Vault is a feature of ansible that allows you to keep sensitive data such as passwords or keys in encrypted files, rather than as plaintext in playbooks or roles. These vault files can then be distributed or placed in source control.</p><h4 id="encrypt-a-file-using-ansible-vault"><a href="#encrypt-a-file-using-ansible-vault" class="headerlink" title="encrypt a file using ansible-vault"></a>encrypt a file using ansible-vault</h4><p>I created a vars file to test ansible-vault command</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">port: 443</span><br><span class="line">greeting: "greeting from vars file"</span><br><span class="line">ansible_user: pi</span><br></pre></td></tr></table></figure><a id="more"></a><p>the command of ansible vault<br><code>ansible-vault encrypt filename</code> Encrypt YAML file</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/ansible-vault/vars-file$ ansible-vault encrypt vars.yml </span><br><span class="line">New Vault password: </span><br><span class="line">Confirm New Vault password: </span><br><span class="line">Encryption successful</span><br><span class="line"></span><br><span class="line">$ANSIBLE_VAULT;1.1;AES256</span><br><span class="line">32616166663766383930373164386335626133313233303037306363306135306534613764666663</span><br><span class="line">6335643762616561393434613331303961373863393035340a376132363132623933623535663933</span><br><span class="line">33393831666662613736616635626366383136333832633566363439343835343637663734663132</span><br><span class="line">3739313466386339380a636633383133356165343335326561626338356561633432646361636639</span><br><span class="line">38653737613034656134353830386239666138653734373535363033303033343232306664343330</span><br><span class="line">32383139313865353930303136383636363839346338326534656663366637373834393063656236</span><br><span class="line">363635613164316232346637303564663239</span><br></pre></td></tr></table></figure><p>then decrypt this file</p><p><code>ansible-vault decrypt filename</code> Decrypt a vault encrypted file</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/ansible-vault/vars-file$ ansible-vault decrypt vars.yml </span><br><span class="line">Vault password: </span><br><span class="line">Decryption successful</span><br><span class="line"></span><br><span class="line">port: 443</span><br><span class="line">greeting: "greeting from vars file"</span><br><span class="line">ansible_user: pi</span><br></pre></td></tr></table></figure><p>other commands:<br><code>ansible-vault view filename</code> View a vault encrypted file with inputting the password<br><code>ansible-vault edit filename</code> Edit a vault encrypted file with password, and without decrypt and encrypt</p><h4 id="argument-ask-vault-pass"><a href="#argument-ask-vault-pass" class="headerlink" title="argument --ask-vault-pass"></a>argument <code>--ask-vault-pass</code></h4><p>after encrypt the vars file, I am going to print the variable of “ansible_usr” in varsfile, following is the playbook:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">tasks: </span><br><span class="line"> - name:</span><br><span class="line"> debug:</span><br><span class="line"> msg: "{{ ansible_user }}"</span><br></pre></td></tr></table></figure><p>because the varsfile is encrypted, the playbook can’t read it without password.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/ansible-vault$ ansible-playbook site.yml</span><br><span class="line">ERROR! Attempting to decrypt but no vault secrets found</span><br></pre></td></tr></table></figure><p>So I need give an argument <code>--ask-vault-pass</code> after <code>ansible-playbook</code> command to let user input the password to decrypt the file.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/ansible-vault$ ansible-playbook site.yml --ask-vault-pass</span><br><span class="line">Vault password: </span><br><span class="line"></span><br><span class="line">PLAY [test ansible vault] **********************************************************************************************************************************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [debug] ***********************************************************************************************************************************************************************************************************************************</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "msg": "pi"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ansible-node1 : ok=1 changed=0 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>With this argument, playbook works well.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part5: how to use module - command module and register and return value of module</title>
<link href="/2020/06/02/Ansible-part5-how-to-use-module/"/>
<url>/2020/06/02/Ansible-part5-how-to-use-module/</url>
<content type="html"><![CDATA[<p>In this session, I am going to talk about command module.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test command module</span><br><span class="line"> command: cat /etc/hosts</span><br></pre></td></tr></table></figure><p>the playbook is pretty simple, let’s look at output</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/command-modules$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test command module] ******************************************************</span><br><span class="line"></span><br><span class="line">TASK [test command module] ******************************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">PLAY RECAP **********************************************************************</span><br><span class="line">ansible-node1 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=1 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>It shows successfully exectute ansible tasks, but I didn’t see the context of /etc/hosts.</p><p>If I want to see the output, I am supposed to let output to be a variable, it looks like following:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test command module</span><br><span class="line"> command: cat /etc/hosts</span><br><span class="line"> register: output</span><br><span class="line"> - name: print the output of command</span><br><span class="line"> debug:</span><br><span class="line"> msg: "{{ output }}"</span><br></pre></td></tr></table></figure><p>after adding register, I can see the /etc/hosts now.</p><a id="more"></a><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/command-modules$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test command module] ******************************************************</span><br><span class="line"></span><br><span class="line">TASK [test command module] ******************************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [print the output of command] **********************************************</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "msg": {</span><br><span class="line"> "ansible_facts": {</span><br><span class="line"> "discovered_interpreter_python": "/usr/bin/python"</span><br><span class="line"> }, </span><br><span class="line"> "changed": true, </span><br><span class="line"> "cmd": [</span><br><span class="line"> "cat", </span><br><span class="line"> "/etc/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "delta": "0:00:00.013819", # this is the time of exectute this ansible task </span><br><span class="line"> "end": "2020-06-03 04:02:04.391584", </span><br><span class="line"> "failed": false, </span><br><span class="line"> "rc": 0, </span><br><span class="line"> "start": "2020-06-03 04:02:04.377765", </span><br><span class="line"> "stderr": "", </span><br><span class="line"> "stderr_lines": [], </span><br><span class="line"> "stdout": "127.0.0.1\tlocalhost\n::1\t\tlocalhost ", </span><br><span class="line"> "stdout_lines": [</span><br><span class="line"> "127.0.0.1\tlocalhost", </span><br><span class="line"> ], </span><br><span class="line"> "warnings": [</span><br><span class="line"> "Platform linux on host ansible-node1 is using the discovered Python interpreter at /usr/bin/python, but future installation of another Python interpreter could change this. See https://docs.ansible.com/ansible/2.9/reference_appendices/interpreter_discovery.html for more information."</span><br><span class="line"> ]</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": {</span><br><span class="line"> "ansible_facts": {</span><br><span class="line"> "discovered_interpreter_python": "/usr/bin/python"</span><br><span class="line"> }, </span><br><span class="line"> "changed": true, </span><br><span class="line"> "cmd": [</span><br><span class="line"> "cat", </span><br><span class="line"> "/etc/hosts"</span><br><span class="line"> ], </span><br><span class="line"> "delta": "0:00:00.011817", </span><br><span class="line"> "end": "2020-06-02 21:02:05.042380", </span><br><span class="line"> "failed": false, </span><br><span class="line"> "rc": 0, </span><br><span class="line"> "start": "2020-06-02 21:02:05.030563", </span><br><span class="line"> "stderr": "", </span><br><span class="line"> "stderr_lines": [], </span><br><span class="line"> "stdout": "127.0.0.1 localhost localhost.localdomain localhost4 ", </span><br><span class="line"> "stdout_lines": [</span><br><span class="line"> "127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4", </span><br><span class="line"> ]</span><br><span class="line"> }</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP **********************************************************************</span><br><span class="line">ansible-node1 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=2 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>In this output I can see a lot of default return value of command modules, for example, rc, stderr, stdout etc, but delta, start and end are command module’s specific return values.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part4: how to use modules - net tools get url</title>
<link href="/2020/06/02/Ansible-part4-how-to-use-modules-net-tools/"/>
<url>/2020/06/02/Ansible-part4-how-to-use-modules-net-tools/</url>
<content type="html"><![CDATA[<p>If I am going to download an python archived file from python.org, I need to use net tools and file module to achieve my goal.</p><h4 id="Net-tools-module-get-url"><a href="#Net-tools-module-get-url" class="headerlink" title="Net tools module: get_url"></a>Net tools module: get_url</h4><p>playbook:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: mkdir</span><br><span class="line"> file:</span><br><span class="line"> path: /tmp/python-download/ # use file module to create directory name /tmp/python-download</span><br><span class="line"> state: directory</span><br><span class="line"> - name: get url</span><br><span class="line"> get_url:</span><br><span class="line"> url: https://www.python.org/ftp/python/3.8.3/Python-3.8.3.tgz # the download url</span><br><span class="line"> dest: /tmp/python-download/</span><br></pre></td></tr></table></figure><p>output:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/net-tools-modules$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test net tool get_url] ****************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] **********************************************************</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [mkdir] ********************************************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [get url] ******************************************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">PLAY RECAP **********************************************************************</span><br><span class="line">ansible-node1 : ok=3 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=3 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>we can check if the python file has been downloaded.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/net-tools-modules$ ansible all -m shell -a "ls /tmp/python-download"</span><br><span class="line">ansible-node1 | CHANGED | rc=0 >></span><br><span class="line">Python-3.8.3.tgz</span><br><span class="line">cube4200 | CHANGED | rc=0 >></span><br><span class="line">Python-3.8.3.tgz</span><br></pre></td></tr></table></figure><p>It shows Python-3.8.3.tgz already downloaded.</p><h4 id="file-modules-unarchive"><a href="#file-modules-unarchive" class="headerlink" title="file modules: unarchive"></a>file modules: unarchive</h4><p>In order to unarchive Python-3.8.3.tgz, I need to use unarchive module</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">- name: unarchive compress file</span><br><span class="line"> unarchive:</span><br><span class="line"> src: /tmp/python-download/Python-3.8.3.tgz</span><br><span class="line"> dest: /etc/tmp</span><br><span class="line"> remote_src: yes #indicate the archived file is already on the remote system and not local to the Ansible controller.</span><br></pre></td></tr></table></figure><p>Notice: <code>remote_src</code> default is no, which means ansible is going to find Python-3.8.3.tgz at my local machine, but actually this file already is in remote, so I need to switch to yes.</p><p>output is:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/net-tools-modules$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">TASK [unarchive compress file] *****************************************************************************************************************************************************************************************************************</span><br><span class="line">changed: [cube4200]</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************************************************************************************************************************************</span><br><span class="line">ansible-node1 : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=4 changed=1 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line"></span><br><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/net-tools-modules$ ansible all -m shell -a "ls /etc/tmp"</span><br><span class="line">cube4200 | CHANGED | rc=0 >></span><br><span class="line">Python-3.8.3</span><br><span class="line">ansible-node1 | CHANGED | rc=0 >></span><br><span class="line">Python-3.8.3</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible-part3: how to use modules - yum apt package</title>
<link href="/2020/05/27/Ansible-part3-how-to-use-modules/"/>
<url>/2020/05/27/Ansible-part3-how-to-use-modules/</url>
<content type="html"><![CDATA[<h4 id="packaging-module-yum"><a href="#packaging-module-yum" class="headerlink" title="packaging module: yum"></a>packaging module: yum</h4><p>yum is CentOS Linux package management tool.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: upgrade all packages</span><br><span class="line"> yum:</span><br><span class="line"> name: '*' # update all packages</span><br><span class="line"> state: latest</span><br><span class="line"> - name: install a package</span><br><span class="line"> yum:</span><br><span class="line"> name: git</span><br><span class="line"> state: latest</span><br></pre></td></tr></table></figure><p>output is:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/packaging_modules/inventory$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test packageing modules] **************************************************</span><br><span class="line"></span><br><span class="line">TASK [upgrade all packages] *****************************************************</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [install a package] ********************************************************</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">PLAY RECAP **********************************************************************</span><br><span class="line">cube4200 : ok=2 changed=2 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0 </span><br><span class="line">[anna@cube4200 ~]$ which git</span><br><span class="line">/usr/bin/git</span><br><span class="line">[anna@cube4200 ~]$ git --version</span><br><span class="line">git version 1.8.3.1</span><br></pre></td></tr></table></figure><p>Notice: the update task took long time to update all packages.</p><h4 id="packaging-module-apt"><a href="#packaging-module-apt" class="headerlink" title="packaging module: apt"></a>packaging module: apt</h4><p>apt is Ubuntu package management tool.</p><p>In this example, I am going to install git in two different linux, one use yum on CentOS, anther one use apt on Debian. So I need to add some condition to let ansible know which node should use yum or apt.<br>Before running site.yml I can use <code>ansible web1 -m gather_facts |grep distribution</code> to see the OS of nodes.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test yum module</span><br><span class="line"> yum:</span><br><span class="line"> name: git</span><br><span class="line"> state: latest</span><br><span class="line"> when: ansible_facts['distribution'] == "CentOS" # this is from gather_facts</span><br><span class="line"> - name: test apt module</span><br><span class="line"> apt:</span><br><span class="line"> name: git</span><br><span class="line"> state: latest</span><br><span class="line"> when: ansible_facts['distribution'] == "Debian"</span><br></pre></td></tr></table></figure><p>During running, I can see ansible will judge which command (apt or yum) to use.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/packaging_modules/inventory$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [test packageing modules] **************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] **********************************************************</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test yum module] **********************************************************</span><br><span class="line">skipping: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [test apt module] **********************************************************</span><br><span class="line">skipping: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">PLAY RECAP **********************************************************************</span><br><span class="line">ansible-node1 : ok=2 changed=0 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0 </span><br><span class="line">cube4200 : ok=2 changed=1 unreachable=0 failed=0 skipped=1 rescued=0 ignored=0</span><br></pre></td></tr></table></figure><p>cube4200 uses CentOS, so when testing apt command, it is skipped; ansible_node1 uses Debian, which is skipped when testing yum.</p><h4 id="packaging-module-package"><a href="#packaging-module-package" class="headerlink" title="packaging module: package"></a>packaging module: package</h4><p>package module is more general and simplify the installing packages</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test package module</span><br><span class="line"> package:</span><br><span class="line"> name: git</span><br><span class="line"> state: latest</span><br></pre></td></tr></table></figure><p>Notice: sometime package will have different name in different OS, under this situation, package module will not work, I still need to use yum or apt with gather_facts conditon to install packages.</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part 2: how to use modules - debug copy file template ping gather-facts user group</title>
<link href="/2020/05/23/Ansible-part-2-examples-of-how-to-use-modules/"/>
<url>/2020/05/23/Ansible-part-2-examples-of-how-to-use-modules/</url>
<content type="html"><![CDATA[<h4 id="Ansible-configuration"><a href="#Ansible-configuration" class="headerlink" title="Ansible configuration"></a>Ansible configuration</h4><p>Before running ansible,I am going to write a default inventory which will be used when I am running site.yml.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[defaults]</span><br><span class="line">inventory = hosts</span><br></pre></td></tr></table></figure><h4 id="Ansible-commands"><a href="#Ansible-commands" class="headerlink" title="Ansible commands"></a>Ansible commands</h4><p>Instead of using ansible playbook, I can also use ansible command to run automation, for example, I am going to check remote nodes directories, I can use ansible command to do that directly, no need to login romote node to check.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/files-modules/inventory$ ansible all -m shell -a "ls /etc/tmp"</span><br><span class="line">ansible-node1 | SUCCESS | rc=0 >></span><br><span class="line">ansible-file-test.txt</span><br><span class="line">ansible-file-test.txt.16075.2020-05-24@22:58:43~</span><br><span class="line"></span><br><span class="line">cube4200 | SUCCESS | rc=0 >></span><br><span class="line">ansible-file-test.txt</span><br><span class="line">ansible-file-test.txt.11629.2020-05-24@15:58:44~</span><br></pre></td></tr></table></figure><p>The command I used is <code>ansible all -m shell -a "ls /etc/tmp"</code></p><h4 id="some-examples-of-how-to-use-modules"><a href="#some-examples-of-how-to-use-modules" class="headerlink" title="some examples of how to use modules"></a>some examples of how to use modules</h4><h5 id="Utilities-modules-gt-debug-gt-print-statements-during-execution"><a href="#Utilities-modules-gt-debug-gt-print-statements-during-execution" class="headerlink" title="Utilities modules -> debug -> print statements during execution"></a>Utilities modules -> debug -> print statements during execution</h5><p>In this module, only three parameters.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">- name: hello world</span><br><span class="line"> hosts: localhost</span><br><span class="line"> tasks:</span><br><span class="line"> - name: hello world</span><br><span class="line"> debug:</span><br><span class="line"> msg: "hello ansible"</span><br><span class="line"> verbosity: 0 # 0 is default, 1-3 parameters will skip the message</span><br></pre></td></tr></table></figure><a id="more"></a><p>running it</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/ansible-code/inventory$ ansible-playbook helloworld.yml </span><br><span class="line"> [WARNING]: * Failed to parse /etc/ansible/hosts with yaml plugin: YAML inventory has invalid structure, it should be</span><br><span class="line">a dictionary, got: <class 'ansible.parsing.yaml.objects.AnsibleUnicode'></span><br><span class="line"></span><br><span class="line"> [WARNING]: * Failed to parse /etc/ansible/hosts with ini plugin: /etc/ansible/hosts:44: Expected key=value host</span><br><span class="line">variable assignment, got: ansible-controller</span><br><span class="line"></span><br><span class="line"> [WARNING]: Unable to parse /etc/ansible/hosts as an inventory source</span><br><span class="line"></span><br><span class="line"> [WARNING]: No inventory was parsed, only implicit localhost is available</span><br><span class="line"></span><br><span class="line"> [WARNING]: provided hosts list is empty, only localhost is available. Note that the implicit localhost does not match</span><br><span class="line">'all'</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">PLAY [hello world] *****************************************************************************************************</span><br><span class="line"></span><br><span class="line">TASK [Gathering Facts] *************************************************************************************************</span><br><span class="line">ok: [localhost]</span><br><span class="line"></span><br><span class="line">TASK [hello world] *****************************************************************************************************</span><br><span class="line">ok: [localhost] => {</span><br><span class="line"> "msg": "hello ansible"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************</span><br><span class="line">localhost : ok=2 changed=0 unreachable=0 failed=0</span><br></pre></td></tr></table></figure><h5 id="files-modules-copy-and-file"><a href="#files-modules-copy-and-file" class="headerlink" title="files modules: copy and file"></a>files modules: copy and file</h5><p>I am going to create a directory and copy some files to remote nodes, so I am going to use two modules, copy and file which will create directory.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">- name: file module</span><br><span class="line"> hosts: all</span><br><span class="line"> gather_facts: no</span><br><span class="line"> become: yes # sudo to root</span><br><span class="line"></span><br><span class="line"> tasks:</span><br><span class="line"> - name: mkdir</span><br><span class="line"> file:</span><br><span class="line"> path: /etc/tmp/ # etc dir needs root right</span><br><span class="line"> state: directory</span><br><span class="line"></span><br><span class="line"> - name: copy files from local to remote</span><br><span class="line"> copy:</span><br><span class="line"> src: files/ansible-file-test.txt</span><br><span class="line"> dest: /etc/tmp/ansible-file-test.txt</span><br><span class="line"> backup: yes # will keep the same name file not be overwrite,give file a time tag</span><br></pre></td></tr></table></figure><p>after running</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/files-modules/invento$ ansible-playbook site.yml</span><br><span class="line"></span><br><span class="line">PLAY [file module] *******************************************************************</span><br><span class="line"></span><br><span class="line">TASK [mkdir] *************************************************************************</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line">ok: [cube4200]</span><br><span class="line"></span><br><span class="line">TASK [copy files from local to remote] ***********************************************</span><br><span class="line">changed: [ansible-node1]</span><br><span class="line">changed: [cube4200]</span><br><span class="line"></span><br><span class="line">PLAY RECAP ***************************************************************************</span><br><span class="line">ansible-node1 : ok=2 changed=1 unreachable=0 failed=0 </span><br><span class="line">cube4200 : ok=2 changed=1 unreachable=0 failed=0</span><br></pre></td></tr></table></figure><p>file modules create a dir named tmp under /etc, since I sudo to root, so I can create dir under etc. after dir is created, I copied my local file named ansible-file-test.txt to node’ destination dir.</p><h5 id="template-modules-template"><a href="#template-modules-template" class="headerlink" title="template modules: template"></a>template modules: template</h5><p>template is like a different copy, in which data is got by variables.<br>Usaually template file is wrote by jinja, so the end of file is .j2.<br>following is a template file named template-demo.j2</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[default]</span><br><span class="line">ansible_user = {{ ansible_user }}</span><br><span class="line">[demo]</span><br><span class="line">{% for ip in range(201, 220) %}</span><br><span class="line">{{ host_prefix }}{{ "%02d" | format(id-200)}}-{{ ip_prefix}}.{{ ip }}</span><br><span class="line">{% endfor %}</span><br></pre></td></tr></table></figure><p>this is template playbook</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">- name: template test</span><br><span class="line"> template:</span><br><span class="line"> src: templates/template-demo.j2</span><br><span class="line"> dest: /etc/tmp/test.cfg</span><br></pre></td></tr></table></figure><h5 id="system-module-ping"><a href="#system-module-ping" class="headerlink" title="system module: ping"></a>system module: ping</h5><p>There are two ways to ping hosts, first use command line<br><code>ansible -m ping all -i inventory</code></p><p>second ways is to use playbook</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: test ping moudle</span><br><span class="line"> ping:</span><br></pre></td></tr></table></figure><p>both work as following:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/Desktop/ansible-code/inventory/modules/system_modules/inventory$ ansible-playbook site.yml </span><br><span class="line"></span><br><span class="line">PLAY [system module] ************************************************************</span><br><span class="line"></span><br><span class="line">TASK [test ping moudle] *********************************************************</span><br><span class="line">ok: [cube4200]</span><br><span class="line">ok: [ansible-node1]</span><br><span class="line"></span><br><span class="line">PLAY RECAP **********************************************************************</span><br><span class="line">ansible-node1 : ok=1 changed=0 unreachable=0 failed=0 </span><br><span class="line">cube4200 : ok=1 changed=0 unreachable=0 failed=0</span><br></pre></td></tr></table></figure><h5 id="system-module-gather-facts"><a href="#system-module-gather-facts" class="headerlink" title="system module: gather_facts"></a>system module: gather_facts</h5><p>In command line, use <code>ansible -m gather_facts -i inventory --tree ./facts</code>, which will get all info of nodes and save it to file which is named facts. If I need the info from facts I can just call it in playbook. For example, I am going to print out the nodes’s distribution.</p><p>Notice: the default of gather_facts is yes, which means, it will get all info if you did not specify it is No.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: print facts</span><br><span class="line"> debug:</span><br><span class="line"> msg: "{{ ansible_distribution }}"</span><br></pre></td></tr></table></figure><p>after running</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">TASK [print facts] **************************************************************</span><br><span class="line">ok: [cube4200] => {</span><br><span class="line"> "msg": "CentOS"</span><br><span class="line">}</span><br><span class="line">ok: [ansible-node1] => {</span><br><span class="line"> "msg": "Debian"</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h5 id="system-module-user-and-group"><a href="#system-module-user-and-group" class="headerlink" title="system module: user and group"></a>system module: user and group</h5><p>manage user and group in linux, such as add new user and group or delete them.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">tasks:</span><br><span class="line"> - name: create group</span><br><span class="line"> group:</span><br><span class="line"> name: ansible_demo</span><br><span class="line"> state: absent # this is default state</span><br><span class="line"> - name: create user</span><br><span class="line"> user:</span><br><span class="line"> name: demo</span><br><span class="line"> - name: delete user</span><br><span class="line"> user:</span><br><span class="line"> name: demo</span><br><span class="line"> state: absent # will not delete home dir</span><br><span class="line"> remove: yes # add this command will delete user home dir</span><br></pre></td></tr></table></figure><p>to be continued</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Ansible part 1: deploy enviroment and most important concepts</title>
<link href="/2020/05/21/Ansible-deploy-enviroment-and-concept/"/>
<url>/2020/05/21/Ansible-deploy-enviroment-and-concept/</url>
<content type="html"><![CDATA[<h2 id="what-is-Ansible"><a href="#what-is-Ansible" class="headerlink" title="what is Ansible"></a>what is Ansible</h2><p>Ansible is an IT automation tool. It can configure systems, deploy software, and orchestrate more advanced IT tasks such as continuous deployments or zero downtime rolling updates.</p><p>There are three main parts in Ansible: inventory, playbook and modules.</p><h3 id="Inventory"><a href="#Inventory" class="headerlink" title="Inventory"></a>Inventory</h3><p>Inventory is a list of managed nodes. An inventory file is also sometimes called a “hostfile”. Your inventory can specify information like IP address for each managed node. An inventory can also organize managed nodes, creating and nesting groups for easier scaling.</p><p>For my enviroment, there are two nodes:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[web1]</span><br><span class="line">ansible-node1 ansible_connection=ssh ansible_user=pi</span><br><span class="line">[web2]</span><br><span class="line">cube4200 ansible_connection=ssh ansible_user=anna</span><br></pre></td></tr></table></figure><h3 id="Playbooks"><a href="#Playbooks" class="headerlink" title="Playbooks"></a>Playbooks</h3><p>Playbooks are Ansible’s configuration, deployment and orchestration language. Each playbook is composed of one or more ‘plays’ in a list.</p><p>Playsbook is yaml file. there are listed tasks you want ansible to work.<br>Playbook is the most important file in ansible.</p><p>In playbooks, there are some important concepts, such as variables, variables files, loops.</p><h4 id="variables"><a href="#variables" class="headerlink" title="variables"></a>variables</h4><p><a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_variables.html" target="_blank" rel="noopener">playbook variables</a> can be wrote directly in playbook or in variable files. all kinds variabls have their precendence during playbook exectued. </p><a id="more"></a><h4 id="Loops"><a href="#Loops" class="headerlink" title="Loops"></a>Loops</h4><p><code>with_items</code> is used in single loop.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">- name: hello loop</span><br><span class="line"> hosts: localhost</span><br><span class="line"> gather_facts: no</span><br><span class="line"> vars:</span><br><span class="line"> test:</span><br><span class="line"> - test1</span><br><span class="line"> - test2</span><br><span class="line"> - test3</span><br><span class="line"> - test4</span><br><span class="line"> tasks:</span><br><span class="line"> - name: Test loop</span><br><span class="line"> debug:</span><br><span class="line"> msg: "{{ item }}"</span><br><span class="line"> with_items: "{{ test }}"</span><br></pre></td></tr></table></figure><p>after runing</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/ansible-code/inventory/loop$ ansible-playbook helloloop.yml </span><br><span class="line"></span><br><span class="line">TASK [Test loop] *******************************************************************************************************</span><br><span class="line">ok: [localhost] => (item=None) => {</span><br><span class="line"> "msg": "test1"</span><br><span class="line">}</span><br><span class="line">ok: [localhost] => (item=None) => {</span><br><span class="line"> "msg": "test2"</span><br><span class="line">}</span><br><span class="line">ok: [localhost] => (item=None) => {</span><br><span class="line"> "msg": "test3"</span><br><span class="line">}</span><br><span class="line">ok: [localhost] => (item=None) => {</span><br><span class="line"> "msg": "test4"</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************</span><br><span class="line">localhost : ok=1 changed=0 unreachable=0 failed=0</span><br></pre></td></tr></table></figure><p><code>with_nested</code> is used in multiple loops.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">- name: nested loop</span><br><span class="line"> hosts: localhost</span><br><span class="line"> gather_facts: no</span><br><span class="line"> vars:</span><br><span class="line"> test:</span><br><span class="line"> - test1</span><br><span class="line"> - test2</span><br><span class="line"> - test3</span><br><span class="line"> - test4</span><br><span class="line"> demo:</span><br><span class="line"> - demo1</span><br><span class="line"> - demo2</span><br><span class="line"> - demo3</span><br><span class="line"> xyz:</span><br><span class="line"> - xyz1</span><br><span class="line"> - xyz2 </span><br><span class="line"> tasks:</span><br><span class="line"> - name: Test loop</span><br><span class="line"> debug:</span><br><span class="line"> msg: "{{ item[0] }}, {{ item[1] }}, {{ item[2] }}"</span><br><span class="line"> with_nested:</span><br><span class="line"> - "{{ test }}"</span><br><span class="line"> - "{{ demo }}"</span><br><span class="line"> - "{{ xyz }}"</span><br></pre></td></tr></table></figure><p>after runing, for loop three iteration and it will show 24 pieces of message.</p><p>Notice: yml file is very strict with indention. BE CARE!</p><h4 id="condition-in-playbook"><a href="#condition-in-playbook" class="headerlink" title="condition in playbook"></a>condition in playbook</h4><p><code>when</code> is the key word in playbook if you want to put some condition in it.</p><p>for example,</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">- name: test conditon</span><br><span class="line"> hosts: localhost</span><br><span class="line"> gather_facts: no</span><br><span class="line"> vars:</span><br><span class="line"> seq:</span><br><span class="line"> - 1</span><br><span class="line"> - 2</span><br><span class="line"> - 3</span><br><span class="line"> - 4</span><br><span class="line"> tasks:</span><br><span class="line"> - name: test loop and when</span><br><span class="line"> debug:</span><br><span class="line"> msg: "{{ item }}"</span><br><span class="line"> with_items: "{{ seq }}"</span><br><span class="line"> when: item >=3</span><br></pre></td></tr></table></figure><p>after running,</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">anna@ansible-controller:~/ansible-code/inventory/condition$ ansible-playbook when.yml </span><br><span class="line">PLAY [test conditon] ***************************************************************************************************</span><br><span class="line">TASK [test loop and when] **********************************************************************************************</span><br><span class="line">skipping: [localhost] => (item=None) </span><br><span class="line">skipping: [localhost] => (item=None) </span><br><span class="line">ok: [localhost] => (item=None) => {</span><br><span class="line"> "msg": 3</span><br><span class="line">}</span><br><span class="line">ok: [localhost] => (item=None) => {</span><br><span class="line"> "msg": 4</span><br><span class="line">}</span><br><span class="line"></span><br><span class="line">PLAY RECAP *************************************************************************************************************</span><br><span class="line">localhost : ok=1 changed=0 unreachable=0 failed=0</span><br></pre></td></tr></table></figure><p>Only item which is greater than 2 can be exectued, so seq 1 and seq2 will be skipped.</p><p>Notice: “and” and “or” in condition, look up the documents <a href="https://docs.ansible.com/ansible/latest/user_guide/playbooks_conditionals.html" target="_blank" rel="noopener">ansible conditon syntax</a></p><h3 id="Modules"><a href="#Modules" class="headerlink" title="Modules"></a>Modules</h3><p><a href="https://docs.ansible.com/ansible/latest/modules/modules_by_category.html" target="_blank" rel="noopener">ansible modules</a> are units of code Ansible executes. Each module has a particular use, from administering users on a specific type of database to managing VLAN interfaces on a specific type of network device. You can invoke a single module with a task, or invoke several different modules in a playbook.Ansible modules are like the function which can help you to automation.</p><p>To be continued…</p>]]></content>
<tags>
<tag> Ansible </tag>
</tags>
</entry>
<entry>
<title>Travis CI: push docker manifest to dockerhub</title>
<link href="/2020/05/11/Travis-CI-push-docker-manifest-to-dockerhub/"/>
<url>/2020/05/11/Travis-CI-push-docker-manifest-to-dockerhub/</url>
<content type="html"><![CDATA[<p>I am going to use Travis to continue create docker multiarch images, then push back to dockerhub repo.</p><h2 id="create-a-repo-in-my-github-account-named-travis-CICD-and-add-it-to-Travis-GitHub-Apps-Integration"><a href="#create-a-repo-in-my-github-account-named-travis-CICD-and-add-it-to-Travis-GitHub-Apps-Integration" class="headerlink" title="create a repo in my github account named travis-CICD and add it to Travis GitHub Apps Integration"></a>create a repo in my github account named <code>travis-CICD</code> and add it to <a href="http://travis-ci.com" target="_blank" rel="noopener">Travis</a> GitHub Apps Integration</h2><img src="/2020/05/11/Travis-CI-push-docker-manifest-to-dockerhub/travis1.png" class=""><h2 id="under-the-travis-CICD-repo-create-a-travis-yml-file"><a href="#under-the-travis-CICD-repo-create-a-travis-yml-file" class="headerlink" title="under the travis-CICD repo, create a .travis.yml file"></a>under the travis-CICD repo, create a .travis.yml file</h2><p>Since github uses ruby, I don’t need to specify the language. For this project, I am going to work on docker, so I need to start docker.<br>I am going to use two docker images to create a multiarch using <code>docker manifest create</code> command.<br>Note: Because manifest is an experimental feature, I need to add <code>DOCKER_CLI_EXPERIMENTAL=enabled</code> in .travis.yml file.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">sudo: required</span><br><span class="line">addons:</span><br><span class="line"> apt:</span><br><span class="line"> update: true</span><br><span class="line"> packages:</span><br><span class="line"> - docker-ce</span><br><span class="line">services:</span><br><span class="line">- docker</span><br><span class="line">before_install:</span><br><span class="line">script:</span><br><span class="line">- docker pull treehouses/turtleblocksjs-tags:nginx-x86</span><br><span class="line">- docker pull treehouses/turtleblocksjs-tags:nginx-rpi</span><br><span class="line">- export DOCKER_CLI_EXPERIMENTAL=enabled</span><br><span class="line">- docker manifest create vmnet8/turtleblocksjs:annarocks2 treehouses/turtleblocksjs-tags:nginx-x86</span><br><span class="line"> treehouses/turtleblocksjs-tags:nginx-rpi</span><br><span class="line">- echo "$DOCKER_PASSWORD" | docker login -u "$DOCKER_USERNAME" --password-stdin</span><br><span class="line">- docker manifest push vmnet8/turtleblocksjs:annarocks2</span><br><span class="line">env:</span><br><span class="line"> global:</span><br></pre></td></tr></table></figure><h2 id="Go-to-dockerhub-to-get-a-new-token"><a href="#Go-to-dockerhub-to-get-a-new-token" class="headerlink" title="Go to dockerhub to get a new token"></a>Go to dockerhub to get a new token</h2><img src="/2020/05/11/Travis-CI-push-docker-manifest-to-dockerhub/travis2.png" class=""><a id="more"></a><h2 id="install-travis-cli-on-my-local"><a href="#install-travis-cli-on-my-local" class="headerlink" title="install travis cli on my local"></a>install travis cli on my local</h2><p>before install travis, I need to solove dependencies.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">sudo apt install ruby-dev libffi-dev make gcc</span><br><span class="line">sudo gem install travis</span><br></pre></td></tr></table></figure><p>to verify it</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~/OLE_intern/travis/travis-CICD$ which travis</span><br><span class="line">/usr/local/bin/travis</span><br><span class="line">anna@ubuntu1804:~$ travis login --com</span><br><span class="line">Shell completion not installed. Would you like to install it now? |y| y</span><br><span class="line">We need your GitHub login to identify you.</span><br><span class="line">This information will not be sent to Travis CI, only to api.github.com.</span><br><span class="line">The password will not be displayed.</span><br><span class="line"></span><br><span class="line">Try running with --github-token or --auto if you don't want to enter your password anyway.</span><br><span class="line"></span><br><span class="line">Username: vmnet8</span><br><span class="line">Password for vmnet8: ********</span><br><span class="line">Successfully logged in as vmnet8!</span><br></pre></td></tr></table></figure><h2 id="create-env-variable-for-travis"><a href="#create-env-variable-for-travis" class="headerlink" title="create env variable for travis"></a>create env variable for travis</h2> <figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">travis env set DOCKER_USERNAME vmnet8</span><br><span class="line">travis env set DOCKER_PASSWORD "******"</span><br></pre></td></tr></table></figure><h2 id="Use-travis-CLI-to-encrypt-data-wiht-the-public-key"><a href="#Use-travis-CLI-to-encrypt-data-wiht-the-public-key" class="headerlink" title="Use travis CLI to encrypt data wiht the public key."></a>Use travis CLI to encrypt data wiht the public key.</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">travis encrypt --pro SOMEVAR="secretvalue"</span><br></pre></td></tr></table></figure><p>It will output a string, I can add it to .travis.yml file</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">travis encrypt SOMEVAR="secretvalue" --add</span><br></pre></td></tr></table></figure><p>It will automaticlly add encrpted data to .travis.yml file</p><h2 id="git-add-travis-yml-file-commit-and-push-to-github"><a href="#git-add-travis-yml-file-commit-and-push-to-github" class="headerlink" title="git add .travis.yml file, commit and push to github"></a>git add .travis.yml file, commit and push to github</h2><p>after the pull request, the travis will build automatic</p><img src="/2020/05/11/Travis-CI-push-docker-manifest-to-dockerhub/travis3.png" class=""><p>exited with 0.<br>Go to dockerhub to verify, it pushed to dockerhub successfully.</p><img src="/2020/05/11/Travis-CI-push-docker-manifest-to-dockerhub/travis4.png" class="">]]></content>
<tags>
<tag> Docker </tag>
<tag> Travis </tag>
</tags>
</entry>
<entry>
<title>Using docker compose yml to run netdata</title>
<link href="/2020/02/12/Using-docker-compose-yml-to-run-netdata/"/>
<url>/2020/02/12/Using-docker-compose-yml-to-run-netdata/</url>
<content type="html"><![CDATA[<p>Want to monitor your system and applications, Netdata is distributed, real-time performance and health monitoring for these. It is a highly-optimized monitoring agent you install on all your systems and containers.</p><p>I tried to use docker image to use netdata.</p><h2 id="Install-netdata-docker-image-and-run-it"><a href="#Install-netdata-docker-image-and-run-it" class="headerlink" title="Install netdata docker image and run it"></a>Install netdata docker image and run it</h2><p>It simply straightforward, just run docker command:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">docker run -d --name=netdata \</span><br><span class="line"> -p 19999:19999 \</span><br><span class="line"> -v /etc/passwd:/host/etc/passwd:ro \</span><br><span class="line"> -v /etc/group:/host/etc/group:ro \</span><br><span class="line"> -v /proc:/host/proc:ro \</span><br><span class="line"> -v /sys:/host/sys:ro \</span><br><span class="line"> -v /etc/os-release:/host/etc/os-release:ro \</span><br><span class="line"> --cap-add SYS_PTRACE \</span><br><span class="line"> --security-opt apparmor=unconfined \</span><br><span class="line"> netdata/netdata</span><br></pre></td></tr></table></figure><p>then go to your browser, <a href="http://yourip:19999" target="_blank" rel="noopener">http://yourip:19999</a>, you can see netdata works.</p><h2 id="use-docker-compose-yml-file-to-run-netdata-service"><a href="#use-docker-compose-yml-file-to-run-netdata-service" class="headerlink" title="use docker compose yml file to run netdata service"></a>use docker compose yml file to run netdata service</h2><h3 id="write-a-shell-script-to-create-a-docker-yml-file-under-srv"><a href="#write-a-shell-script-to-create-a-docker-yml-file-under-srv" class="headerlink" title="write a shell script to create a docker yml file under /srv"></a>write a shell script to create a docker yml file under /srv</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">#!/bin/bash</span><br><span class="line"></span><br><span class="line">#create netdata.yml</span><br><span class="line">mkdir -p /srv/netdata/</span><br><span class="line">{</span><br><span class="line"> echo "version: '3'"</span><br><span class="line"> echo "services:"</span><br><span class="line"> echo " netdata:"</span><br><span class="line"> echo " image: netdata/netdata"</span><br><span class="line"> echo " ports:"</span><br><span class="line"> echo " - 19999:19999"</span><br><span class="line"> echo " cap_add:"</span><br><span class="line"> echo " - SYS_PTRACE"</span><br><span class="line"> echo " security_opt:"</span><br><span class="line"> echo " - apparmor:unconfined"</span><br><span class="line"> echo " volumes:"</span><br><span class="line"> echo " - /etc/passwd:/host/etc/passwd:ro"</span><br><span class="line"> echo " - /etc/os-release:/host/etc/os-release:ro"</span><br><span class="line"> echo " - /etc/group:/host/etc/group:ro"</span><br><span class="line"> echo " - /proc:/host/proc:ro"</span><br><span class="line"> echo " - /sys:/host/sys:ro"</span><br><span class="line">} > /srv/netdata/netdata.yml</span><br></pre></td></tr></table></figure><p>run it, then go to /srv/netdata to check if it exists</p><p>at the same dir to create an autorun file, named netdata_auto</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">netdata_autorun=true</span><br><span class="line"></span><br><span class="line">if [ "$netdata_autorun" = true ]; then</span><br><span class="line"> docker-compose -f /srv/netdata/netdata.yml -p netdata up -d</span><br><span class="line">fi</span><br></pre></td></tr></table></figure><p>let me test</p><p>run shell script first</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">root@rpi-121:/home/pi/git_repo/cli/templates/services/netdata# ./netdata_yml.sh</span><br></pre></td></tr></table></figure><p>go to /srv to verify it created the docker compose yml file</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">root@rpi-121:/home/pi/git_repo/cli/templates/services/netdata# cat /srv/netdata/netdata.yml </span><br><span class="line">version: '3'</span><br><span class="line">services:</span><br><span class="line"> netdata:</span><br><span class="line"> image: netdata/netdata</span><br><span class="line"> ports:</span><br><span class="line"> - 19999:19999</span><br><span class="line"> cap_add:</span><br><span class="line"> - SYS_PTRACE</span><br><span class="line"> security_opt:</span><br><span class="line"> - apparmor:unconfined</span><br><span class="line"> volumes:</span><br><span class="line"> - /etc/passwd:/host/etc/passwd:ro</span><br><span class="line"> - /etc/os-release:/host/etc/os-release:ro</span><br><span class="line"> - /etc/group:/host/etc/group:ro</span><br><span class="line"> - /proc:/host/proc:ro</span><br><span class="line"> - /sys:/host/sys:ro</span><br></pre></td></tr></table></figure><p>then run docker-compose</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">root@rpi-121:/home/pi/git_repo/cli/templates/services/netdata# docker-compose -f /srv/netdata/netdata.yml -p netdata up -d</span><br><span class="line">Creating network "netdata_default" with the default driver</span><br><span class="line">Pulling netdata (netdata/netdata:)...</span><br><span class="line">latest: Pulling from netdata/netdata</span><br><span class="line">2aa204966f9a: Pull complete</span><br><span class="line">8dde373fbef5: Pull complete</span><br><span class="line">042f66a41d56: Pull complete</span><br><span class="line">3526782612f0: Pull complete</span><br><span class="line">c2b930d703fa: Pull complete</span><br><span class="line">9b859bae4175: Pull complete</span><br><span class="line">f3e46298693e: Pull complete</span><br><span class="line">Digest: sha256:b49e1901fd8aae43d950c171c14449b5080a5550faa97594e2d4ccee7ed3aac8</span><br><span class="line">Status: Downloaded newer image for netdata/netdata:latest</span><br><span class="line">Creating netdata_netdata_1 ... done</span><br></pre></td></tr></table></figure><p>check if the docker image is pulled and run</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">root@rpi-121:/home/pi/git_repo/cli/templates/services/netdata# docker images</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">netdata/netdata latest d170142fbe20 19 hours ago 249MB</span><br></pre></td></tr></table></figure><p>it worked well</p><p>go to browser <a href="http://myip:19999" target="_blank" rel="noopener">http://myip:19999</a></p><a id="more"></a><img src="/2020/02/12/Using-docker-compose-yml-to-run-netdata/netdata.png" class="">]]></content>
<tags>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Docker manifest to create multi-arch docker image</title>
<link href="/2020/01/15/Docker-manifest-to-create-multi-arch-docker-image/"/>
<url>/2020/01/15/Docker-manifest-to-create-multi-arch-docker-image/</url>
<content type="html"><![CDATA[<p>go to github.com/treehouses/rpi-nginx dir<br>modifid Dockerfile</p><p>run docker build<br>root@vmnet8cloud:~/git_repo/treehouses/rpi-nginx# docker build -t vmnet8/preplanet:rpi-1.1 .</p><p>root@vmnet8cloud:<del>/git_repo/treehouses/rpi-nginx# docker images<br>REPOSITORY TAG IMAGE ID CREATED SIZE<br>treehouses/rpi-nginx 1.0 1bd2ae323561 3 minutes ago 50.7MB<br>root@vmnet8cloud:</del>/git_repo/treehouses/rpi-nginx# docker run -d treehouses/rpi-nginx:1.0<br>22992681fb717fcf19cd9691dbaf7089b2280e0667881e8d0ce18b9a1a82da70<br>root@vmnet8cloud:~/git_repo/treehouses/rpi-nginx# docker ps -a<br>CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES<br>22992681fb71 treehouses/rpi-nginx:1.0 “/usr/bin/entry.sh n…” 9 seconds ago Up 8 seconds 80/tcp, 443/tcp mystifying_rhodes</p><p>root@vmnet8cloud:~/git_repo/treehouses/rpi-nginx# docker push treehouses/rpi-nginx:1.0<br>The push refers to repository [docker.io/treehouses/rpi-nginx]<br>7023a6605ba3: Pushed<br>64ac289193c5: Pushed<br>4b51db9fecf2: Mounted from vmnet8/preplanet<br>4206eb6099cb: Mounted from vmnet8/preplanet<br>3d13333fb74a: Mounted from vmnet8/preplanet<br>6eca2d86fb5c: Mounted from vmnet8/preplanet<br>9863749692b7: Mounted from vmnet8/preplanet<br>77ffeb9e9842: Mounted from vmnet8/rpi-nginx<br>1.0: digest: sha256:acb7fc62ea8ba61ad58c17f774c35c8c3284a51dda29c575935c732655271c60 size: 1995</p><p>going to create a multi-arch docker hub</p><p>root@vmnet8cloud:~# docker images<br>REPOSITORY TAG IMAGE ID CREATED SIZE<br>treehouses/rpi-nginx 1.0 1bd2ae323561 2 hours ago 50.7MB<br>nginx 1.14.0-alpine 68513e2e4bce 16 months ago 14.4MB</p><p>I need use nginx:1.16.0-alpine to create multi-arch dockerhub image, so</p><p>root@vmnet8cloud:~# docker pull nginx:1.16.0-alpine</p><p>run </p><p>root@vmnet8cloud:~# docker images<br>REPOSITORY TAG IMAGE ID CREATED SIZE<br>treehouses/rpi-nginx 1.0 1bd2ae323561 2 hours ago 50.7MB<br>nginx 1.16.0-alpine 05edf438c0e4 8 months ago 17.6MB<br>nginx 1.14.0-alpine 68513e2e4bce 16 months ago 14.4MB</p><p>In order to create docker manifest list, I need to modify the docker image’s tag</p><p>root@vmnet8cloud:<del># docker tag treehouses/rpi-nginx:1.0 treehouses/nginx:rpi<br>root@vmnet8cloud:</del># docker tag nginx:1.16.0-alpine treehouses/nginx:x86</p><p>After change the tags, it shows new image name</p><p>root@vmnet8cloud:~# docker images<br>REPOSITORY TAG IMAGE ID CREATED SIZE<br>treehouses/nginx rpi 1bd2ae323561 2 hours ago 50.7MB<br>treehouses/rpi-nginx 1.0 1bd2ae323561 2 hours ago 50.7MB<br>nginx 1.16.0-alpine 05edf438c0e4 8 months ago 17.6MB<br>treehouses/nginx x86 05edf438c0e4 8 months ago 17.6MB<br>nginx 1.14.0-alpine 68513e2e4bce 16 months ago 14.4MB</p><p>I need to push these two new images (name) to dockerhub</p><p>root@vmnet8cloud:<del>/git_repo/treehouses/rpi-nginx# docker push treehouses/nginx:rpi<br>root@vmnet8cloud:</del>/git_repo/treehouses/rpi-nginx# docker push treehouses/nginx:x86</p><p>create a multi-arch docker image</p><p>root@vmnet8cloud:~/git_repo/treehouses/rpi-nginx# docker manifest create treehouses/nginx:latest treehouses/nginx:x86 treehouses/nginx:rpi</p><p>Last steps to push this multi-arch to docker hub</p><p>root@vmnet8cloud:~/git_repo/treehouses/rpi-nginx# docker manifest push treehouses/nginx</p>]]></content>
<tags>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Modify a Dockerfile</title>
<link href="/2020/01/13/Modify-a-Dockerfile/"/>
<url>/2020/01/13/Modify-a-Dockerfile/</url>
<content type="html"><![CDATA[<p>I need to rewrite a Dockerfile to create my own Dockerfile.<br>I pull a docker image base from dockerhub first.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line">FROM balenalib/raspberry-pi-alpine:3.9</span><br><span class="line"></span><br><span class="line">LABEL org.opencontainers.image.authors="Tobias Hargesheimer <[email protected]>" \</span><br><span class="line">org.opencontainers.image.title="NGINX" \</span><br><span class="line">org.opencontainers.image.description="AlpineLinux with NGINX on arm arch" \</span><br><span class="line">org.opencontainers.image.licenses="Apache-2.0" \</span><br><span class="line">org.opencontainers.image.url="https://hub.docker.com/r/tobi312/rpi-nginx/" \</span><br><span class="line">org.opencontainers.image.source="https://github.com/Tob1asDocker/rpi-nginx"</span><br><span class="line"></span><br><span class="line">ARG CROSS_BUILD_START=":"</span><br><span class="line">ARG CROSS_BUILD_END=":"</span><br><span class="line"></span><br><span class="line">RUN [ ${CROSS_BUILD_START} ]</span><br><span class="line"></span><br><span class="line">ENV NGINX_VERSION 1.14</span><br><span class="line"></span><br><span class="line">RUN apk --no-cache add nginx>${NGINX_VERSION} \</span><br><span class="line">&& mkdir -p /run/nginx \</span><br><span class="line">&& sed -i "s/ssl_session_cache shared:SSL:2m;/#ssl_session_cache shared:SSL:2m;/g" /etc/nginx/nginx.conf</span><br><span class="line"></span><br><span class="line"># forward request and error logs to docker log collector</span><br><span class="line">RUN ln -sf /dev/stdout /var/log/nginx/access.log \</span><br><span class="line">&& ln -sf /dev/stderr /var/log/nginx/error.log</span><br><span class="line"></span><br><span class="line"># fix: *** stack smashing detected ***: nginx: worker process terminated / [alert] 9#9: worker process *process-id* exited on signal 6</span><br><span class="line">#RUN sed -i "s/worker_processes auto;/worker_processes 1;/g" /etc/nginx/nginx.conf</span><br><span class="line"></span><br><span class="line">EXPOSE 80 443</span><br><span class="line"></span><br><span class="line">CMD ["nginx", "-g", "daemon off;"]</span><br><span class="line"></span><br><span class="line">RUN [ ${CROSS_BUILD_END} ]</span><br></pre></td></tr></table></figure><p>For this image, its base is balenalib/raspberry-pi-alpine:3.9. when I am going to build my own docker image, it has the newest version, but I still use the old one, because it will be more stable for me.</p><p>I add some new command unders this docker base Dockerfile</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line">FROM balenalib/raspberry-pi-alpine:3.9</span><br><span class="line"></span><br><span class="line">LABEL org.opencontainers.image.authors="Tobias Hargesheimer <[email protected]>" \</span><br><span class="line">org.opencontainers.image.title="NGINX" \</span><br><span class="line">org.opencontainers.image.description="AlpineLinux with NGINX on arm arch" \</span><br><span class="line">org.opencontainers.image.licenses="Apache-2.0" \</span><br><span class="line">org.opencontainers.image.url="https://hub.docker.com/r/tobi312/rpi-nginx/" \</span><br><span class="line">org.opencontainers.image.source="https://github.com/Tob1asDocker/rpi-nginx"</span><br><span class="line"></span><br><span class="line">ARG CROSS_BUILD_START=":"</span><br><span class="line">ARG CROSS_BUILD_END=":"</span><br><span class="line"></span><br><span class="line">RUN [ ${CROSS_BUILD_START} ]</span><br><span class="line"></span><br><span class="line">ENV NGINX_VERSION 1.14</span><br><span class="line"></span><br><span class="line">RUN apk --no-cache add nginx>${NGINX_VERSION} \</span><br><span class="line">&& mkdir -p /run/nginx \</span><br><span class="line">&& sed -i "s/ssl_session_cache shared:SSL:2m;/#ssl_session_cache shared:SSL:2m;/g" /etc/nginx/nginx.conf</span><br><span class="line"></span><br><span class="line"># forward request and error logs to docker log collector</span><br><span class="line">RUN ln -sf /dev/stdout /var/log/nginx/access.log \</span><br><span class="line">&& ln -sf /dev/stderr /var/log/nginx/error.log</span><br><span class="line"></span><br><span class="line"># fix: *** stack smashing detected ***: nginx: worker process terminated / [alert] 9#9: worker process *process-id* exited on signal 6</span><br><span class="line">#RUN sed -i "s/worker_processes auto;/worker_processes 1;/g" /etc/nginx/nginx.conf</span><br><span class="line"></span><br><span class="line">EXPOSE 80 443</span><br><span class="line"></span><br><span class="line">CMD ["nginx", "-g", "daemon off;"]</span><br><span class="line"></span><br><span class="line">RUN [ ${CROSS_BUILD_END} ]</span><br><span class="line">RUN apk add --no-cache \</span><br><span class="line"> fcgi=2.4.0-r8 \</span><br><span class="line"> fcgiwrap=1.1.0-r3 \</span><br><span class="line"> spawn-fcgi=1.6.4-r3 \</span><br><span class="line"> ca-certificates=20190108-r0 \</span><br><span class="line"> nghttp2-libs=1.35.1-r0 \</span><br><span class="line"> libssh2=1.8.2-r0 \</span><br><span class="line"> libcurl=7.64.0-r2 \</span><br><span class="line"> curl=7.64.0-r2 \</span><br><span class="line"> oniguruma=6.9.4-r0 \</span><br><span class="line"> jq=1.6-r0</span><br></pre></td></tr></table></figure><p>Notice: the packages need to be installed all have version numbers which will cause the error in the following.</p><p>After modify the Dockerfile, I am going to build my docker image.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/git_repo/preplanet/docker/planet# docker build -t vmnet8/preplanet:rpi-1 .</span><br><span class="line">Sending build context to Docker daemon 4.096kB</span><br><span class="line">Step 1/12 : FROM balenalib/raspberry-pi-alpine:3.9</span><br><span class="line"> ---> d6c5aebf964a</span><br><span class="line">Step 2/12 : LABEL org.opencontainers.image.authors="Tobias Hargesheimer <[email protected]>" org.opencontainers.image.title="NGINX" org.opencontainers.image.description="AlpineLinux with NGINX on arm arch" org.opencontainers.image.licenses="Apache-2.0" org.opencontainers.image.url="https://hub.docker.com/r/tobi312/rpi-nginx/" org.opencontainers.image.source="https://github.com/Tob1asDocker/rpi-nginx"</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 1a8a69474af2</span><br><span class="line">Step 3/12 : ARG CROSS_BUILD_START=":"</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 2ea7d76cc571</span><br><span class="line">Step 4/12 : ARG CROSS_BUILD_END=":"</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 9dac95067f17</span><br><span class="line">Step 5/12 : RUN [ ${CROSS_BUILD_START} ]</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> e1d8ae03657f</span><br><span class="line">Step 6/12 : ENV NGINX_VERSION 1.14</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 684896377216</span><br><span class="line">Step 7/12 : RUN apk --no-cache add nginx>${NGINX_VERSION} && mkdir -p /run/nginx && sed -i "s/ssl_session_cache shared:SSL:2m;/#ssl_session_cache shared:SSL:2m;/g" /etc/nginx/nginx.conf</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> ef20e56136c1</span><br><span class="line">Step 8/12 : RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 8d95b6ad6846</span><br><span class="line">Step 9/12 : EXPOSE 80 443</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> a12bf17fcaea</span><br><span class="line">Step 10/12 : CMD ["nginx", "-g", "daemon off;"]</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 1f6e9eb85851</span><br><span class="line">Step 11/12 : RUN [ ${CROSS_BUILD_END} ]</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 969cc49b4faf</span><br><span class="line">Step 12/12 : RUN apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r0 libssh2=1.8.2-r0 libcurl=7.64.0-r2 curl=7.64.0-r2 oniguruma=6.9.4-r0 jq=1.6-r0</span><br><span class="line"> ---> Running in 910227bcece4</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/armhf/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/armhf/APKINDEX.tar.gz</span><br><span class="line">ERROR: unsatisfiable constraints:</span><br><span class="line"> nghttp2-libs-1.35.1-r1:</span><br><span class="line"> breaks: world[nghttp2-libs=1.35.1-r0]</span><br><span class="line"> satisfies: libcurl-7.64.0-r3[so:libnghttp2.so.14]</span><br><span class="line"> libssh2-1.9.0-r1:</span><br><span class="line"> breaks: world[libssh2=1.8.2-r0]</span><br><span class="line"> satisfies: libcurl-7.64.0-r3[so:libssh2.so.1]</span><br><span class="line"> libcurl-7.64.0-r3:</span><br><span class="line"> breaks: world[libcurl=7.64.0-r2]</span><br><span class="line"> satisfies: curl-7.64.0-r3[so:libcurl.so.4]</span><br><span class="line"> curl-7.64.0-r3:</span><br><span class="line"> breaks: world[curl=7.64.0-r2]</span><br><span class="line">The command '/bin/sh -c apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r0 libssh2=1.8.2-r0 libcurl=7.64.0-r2 curl=7.64.0-r2 oniguruma=6.9.4-r0 jq=1.6-r0' returned a non-zero code: 4</span><br></pre></td></tr></table></figure><p>It shows error.</p><p>I modified the Dockerfile again and move off all packages versions like following</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line">FROM balenalib/raspberry-pi-alpine:3.9</span><br><span class="line"></span><br><span class="line">LABEL org.opencontainers.image.authors="Tobias Hargesheimer <[email protected]>" \</span><br><span class="line">org.opencontainers.image.title="NGINX" \</span><br><span class="line">org.opencontainers.image.description="AlpineLinux with NGINX on arm arch" \</span><br><span class="line">org.opencontainers.image.licenses="Apache-2.0" \</span><br><span class="line">org.opencontainers.image.url="https://hub.docker.com/r/tobi312/rpi-nginx/" \</span><br><span class="line">org.opencontainers.image.source="https://github.com/Tob1asDocker/rpi-nginx"</span><br><span class="line"></span><br><span class="line">ARG CROSS_BUILD_START=":"</span><br><span class="line">ARG CROSS_BUILD_END=":"</span><br><span class="line"></span><br><span class="line">RUN [ ${CROSS_BUILD_START} ]</span><br><span class="line"></span><br><span class="line">ENV NGINX_VERSION 1.14</span><br><span class="line"></span><br><span class="line">RUN apk --no-cache add nginx>${NGINX_VERSION} \</span><br><span class="line">&& mkdir -p /run/nginx \</span><br><span class="line">&& sed -i "s/ssl_session_cache shared:SSL:2m;/#ssl_session_cache shared:SSL:2m;/g" /etc/nginx/nginx.conf</span><br><span class="line"></span><br><span class="line"># forward request and error logs to docker log collector</span><br><span class="line">RUN ln -sf /dev/stdout /var/log/nginx/access.log \</span><br><span class="line">&& ln -sf /dev/stderr /var/log/nginx/error.log</span><br><span class="line"></span><br><span class="line"># fix: *** stack smashing detected ***: nginx: worker process terminated / [alert] 9#9: worker process *process-id* exited on signal 6</span><br><span class="line">#RUN sed -i "s/worker_processes auto;/worker_processes 1;/g" /etc/nginx/nginx.conf</span><br><span class="line"></span><br><span class="line">EXPOSE 80 443</span><br><span class="line"></span><br><span class="line">CMD ["nginx", "-g", "daemon off;"]</span><br><span class="line"></span><br><span class="line">RUN [ ${CROSS_BUILD_END} ]</span><br><span class="line">RUN apk add --no-cache \</span><br><span class="line"> fcgi \</span><br><span class="line"> fcgiwrap \</span><br><span class="line"> spawn-fcgi \</span><br><span class="line"> ca-certificates \</span><br><span class="line"> nghttp2-libs \</span><br><span class="line"> libssh2 \</span><br><span class="line"> libcurl \</span><br><span class="line"> curl \</span><br><span class="line"> oniguruma \</span><br><span class="line"> jq</span><br></pre></td></tr></table></figure><p>then build docker image again</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/git_repo/preplanet/docker/planet# docker build -t vmnet8/preplanet:rpi-1 .</span><br><span class="line">Sending build context to Docker daemon 6.144kB</span><br><span class="line">Step 1/12 : FROM balenalib/raspberry-pi-alpine:3.9</span><br><span class="line"> ---> d6c5aebf964a</span><br><span class="line">Step 2/12 : LABEL org.opencontainers.image.authors="Tobias Hargesheimer <[email protected]>" org.opencontainers.image.title="NGINX" org.opencontainers.image.description="AlpineLinux with NGINX on arm arch" org.opencontainers.image.licenses="Apache-2.0" org.opencontainers.image.url="https://hub.docker.com/r/tobi312/rpi-nginx/" org.opencontainers.image.source="https://github.com/Tob1asDocker/rpi-nginx"</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 1a8a69474af2</span><br><span class="line">Step 3/12 : ARG CROSS_BUILD_START=":"</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 2ea7d76cc571</span><br><span class="line">Step 4/12 : ARG CROSS_BUILD_END=":"</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 9dac95067f17</span><br><span class="line">Step 5/12 : RUN [ ${CROSS_BUILD_START} ]</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> e1d8ae03657f</span><br><span class="line">Step 6/12 : ENV NGINX_VERSION 1.14</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 684896377216</span><br><span class="line">Step 7/12 : RUN apk --no-cache add nginx>${NGINX_VERSION} && mkdir -p /run/nginx && sed -i "s/ssl_session_cache shared:SSL:2m;/#ssl_session_cache shared:SSL:2m;/g" /etc/nginx/nginx.conf</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> ef20e56136c1</span><br><span class="line">Step 8/12 : RUN ln -sf /dev/stdout /var/log/nginx/access.log && ln -sf /dev/stderr /var/log/nginx/error.log</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 8d95b6ad6846</span><br><span class="line">Step 9/12 : EXPOSE 80 443</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> a12bf17fcaea</span><br><span class="line">Step 10/12 : CMD ["nginx", "-g", "daemon off;"]</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 1f6e9eb85851</span><br><span class="line">Step 11/12 : RUN [ ${CROSS_BUILD_END} ]</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 969cc49b4faf</span><br><span class="line">Step 12/12 : RUN apk add --no-cache fcgi fcgiwrap spawn-fcgi ca-certificates nghttp2-libs libssh2 libcurl curl oniguruma jq</span><br><span class="line"> ---> Running in 95d22279f720</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/armhf/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/armhf/APKINDEX.tar.gz</span><br><span class="line">(1/5) Installing fcgi (2.4.0-r8)</span><br><span class="line">(2/5) Installing fcgiwrap (1.1.0-r3)</span><br><span class="line">Executing fcgiwrap-1.1.0-r3.pre-install</span><br><span class="line">(3/5) Installing oniguruma (6.9.4-r0)</span><br><span class="line">(4/5) Installing jq (1.6-r0)</span><br><span class="line">(5/5) Installing spawn-fcgi (1.6.4-r3)</span><br><span class="line">Executing busybox-1.29.3-r10.trigger</span><br><span class="line">OK: 52 MiB in 77 packages</span><br><span class="line">Removing intermediate container 95d22279f720</span><br><span class="line"> ---> 7ee7b73f89ee</span><br><span class="line">Successfully built 7ee7b73f89ee</span><br><span class="line">Successfully tagged vmnet8/preplanet:rpi-1</span><br></pre></td></tr></table></figure><p>After building, test the running container</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">run docer image vmnet8/preplanet:rpi-1</span><br><span class="line">root@vmnet8cloud:~/git_repo/preplanet/docker/planet# docker run -d vmnet8/preplanet:rpi-1</span><br><span class="line">b974aee9b9bb0d6242dcbab12e975585ff1e2d4b887860523b6d0b261863d562</span><br><span class="line">root@vmnet8cloud:~/git_repo/preplanet/docker/planet# docker ps -a</span><br><span class="line">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span><br><span class="line">b974aee9b9bb vmnet8/preplanet:rpi-1 "/usr/bin/entry.sh n…" 9 seconds ago Up 7 seconds 80/tcp, 443/tcp goofy_banach</span><br><span class="line">270cf6e3e185 vmnet8/preplanet:rpi-1 "/usr/bin/entry.sh n…" 2 minutes ago Exited (0) About a minute ago cranky_wing</span><br><span class="line">7259cecd0e74 969cc49b4faf "/bin/sh -c 'apk add…" 11 hours ago Exited (99) 11 hours ago dreamy_brown</span><br><span class="line">910227bcece4 969cc49b4faf "/bin/sh -c 'apk add…" 11 hours ago Exited (4) 11 hours ago modest_wu</span><br><span class="line">d4f32b1bc836 68513e2e4bce "/bin/sh -c 'apk add…" 14 hours ago Exited (6) 14 hours ago frosty_herschel</span><br><span class="line">baa0a10c37a7 324ab2e7954e "/bin/sh -c 'bash ./…" 4 weeks ago Exited (127) 4 weeks ago cranky_herschel</span><br><span class="line">b5ba36510956 vmnet8/preplanet:rpi "/usr/bin/entry.sh n…" 4 weeks ago Up 4 weeks 443/tcp, 0.0.0.0:20006->80/tcp heuristic_chaplygin</span><br><span class="line"></span><br><span class="line"> 201,0-1 Bot</span><br></pre></td></tr></table></figure><p>It shows it works.</p>]]></content>
<tags>
<tag> Docker </tag>
<tag> Dockerfile </tag>
</tags>
</entry>
<entry>
<title>Build multi-arch docker image</title>
<link href="/2019/12/09/Build-multi-arch-docker-image/"/>
<url>/2019/12/09/Build-multi-arch-docker-image/</url>
<content type="html"><![CDATA[<p>In this article I am trying to build a multi-arch image that supports arm and amd64, then I will push it to Docker Hub.</p><h2 id="Build-docker-image"><a href="#Build-docker-image" class="headerlink" title="Build docker image"></a>Build docker image</h2><p>Two dockfiles have different arch.</p><h3 id="Build-an-arm-arch"><a href="#Build-an-arm-arch" class="headerlink" title="Build an arm arch"></a>Build an arm arch</h3><p>Dockfile</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">FROM tobi312/rpi-nginx@sha256:b375cd9a4511f5d7763397c5a4faa1f5c3605507db97075b12582aad2a6871b0</span><br><span class="line">RUN rm -rf /usr/share/nginx/html/*</span><br><span class="line">RUN apk add --no-cache \</span><br><span class="line"> fcgi=2.4.0-r8 \</span><br><span class="line"> fcgiwrap=1.1.0-r3 \</span><br><span class="line"> spawn-fcgi=1.6.4-r3 \</span><br><span class="line"> ca-certificates=20190108-r0 \</span><br><span class="line"> nghttp2-libs=1.35.1-r0 \</span><br><span class="line"> libssh2=1.8.2-r0 \</span><br><span class="line"> libcurl=7.64.0-r2 \</span><br><span class="line"> curl=7.64.0-r2 \</span><br><span class="line"> oniguruma=6.9.4-r0 \</span><br><span class="line"> jq=1.6-r0</span><br></pre></td></tr></table></figure><p>change directory under this dockfile to build image</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/rpi# docker build . -t vmnet8/rpi</span><br><span class="line">Sending build context to Docker daemon 3.072kB</span><br><span class="line">Step 1/3 : FROM tobi312/rpi-nginx@sha256:b375cd9a4511f5d7763397c5a4faa1f5c3605507db97075b12582aad2a6871b0</span><br><span class="line">sha256:b375cd9a4511f5d7763397c5a4faa1f5c3605507db97075b12582aad2a6871b0: Pulling from tobi312/rpi-nginx</span><br><span class="line">6e39823df636: Already exists </span><br><span class="line">38dd00ade3c1: Already exists </span><br><span class="line">c02057c17ddc: Already exists </span><br><span class="line">e6d776bad3ac: Already exists </span><br><span class="line">92ca2fd255df: Already exists </span><br><span class="line">a2cfa1cf1cbb: Already exists </span><br><span class="line">35e3be655ed6: Already exists </span><br><span class="line">f182036f54c2: Already exists </span><br><span class="line">Digest: sha256:b375cd9a4511f5d7763397c5a4faa1f5c3605507db97075b12582aad2a6871b0</span><br><span class="line">Status: Downloaded newer image for tobi312/rpi-nginx@sha256:b375cd9a4511f5d7763397c5a4faa1f5c3605507db97075b12582aad2a6871b0</span><br><span class="line"> ---> 91ed615e0b64</span><br><span class="line">Step 2/3 : RUN rm -rf /usr/share/nginx/html/*</span><br><span class="line"> ---> Running in aa2b4077aa07</span><br><span class="line">Removing intermediate container aa2b4077aa07</span><br><span class="line"> ---> 9412dc8b27c2</span><br><span class="line">Step 3/3 : RUN apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r0 libssh2=1.8.2-r0 libcurl=7.64.0-r2 curl=7.64.0-r2 oniguruma=6.9.4-r0 jq=1.6-r0</span><br><span class="line"> ---> Running in a6902c745dd5</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/armhf/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/armhf/APKINDEX.tar.gz</span><br><span class="line">(1/5) Installing fcgi (2.4.0-r8)</span><br><span class="line">(2/5) Installing fcgiwrap (1.1.0-r3)</span><br><span class="line">Executing fcgiwrap-1.1.0-r3.pre-install</span><br><span class="line">(/5) Installing oniguruma (6.9.4-r0)</span><br><span class="line">(4/5) Installing jq (1.6-r0)</span><br><span class="line">(5/5) Installing spawn-fcgi (1.6.4-r3)</span><br><span class="line">Executing busybox-1.29.3-r10.trigger</span><br><span class="line">OK: 52 MiB in 77 packages</span><br><span class="line">Removing intermediate container a6902c745dd5</span><br><span class="line"> ---> 758ef3313bac</span><br><span class="line">Successfully built 758ef3313bac</span><br><span class="line">Successfully tagged vmnet8/rpi:latest</span><br></pre></td></tr></table></figure><h3 id="Build-an-amd64-arch"><a href="#Build-an-amd64-arch" class="headerlink" title="Build an amd64 arch"></a>Build an amd64 arch</h3><p>Dockerfile</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">FROM nginx:1.16.0-alpine</span><br><span class="line"></span><br><span class="line">RUN rm -rf /usr/share/nginx/html/*</span><br><span class="line">RUN apk add --no-cache \</span><br><span class="line"> fcgi=2.4.0-r8 \</span><br><span class="line"> fcgiwrap=1.1.0-r3 \</span><br><span class="line"> spawn-fcgi=1.6.4-r3 \</span><br><span class="line"> ca-certificates=20190108-r0 \</span><br><span class="line"> nghttp2-libs=1.35.1-r1 \</span><br><span class="line"> libssh2=1.9.0-r1 \</span><br><span class="line"> libcurl=7.64.0-r3 \</span><br><span class="line"> curl=7.64.0-r3 \</span><br><span class="line"> oniguruma=6.9.4-r0 \</span><br><span class="line"> jq=1.6-r0</span><br></pre></td></tr></table></figure><p>build processing</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/template# docker build . -t vmnet8/planet</span><br><span class="line">Sending build context to Docker daemon 2.048kB</span><br><span class="line">Step 1/3 : FROM nginx:1.16.0-alpine</span><br><span class="line"> ---> 05edf438c0e4</span><br><span class="line">Step 2/3 : RUN rm -rf /usr/share/nginx/html/*</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> f7441287619e</span><br><span class="line">Step 3/3 : RUN apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r1 libssh2=1.9.0-r1 libcurl=7.64.0-r3 curl=7.64.0-r3 oniguruma=6.9.4-r0 jq=1.6-r0</span><br><span class="line"> ---> Using cache</span><br><span class="line"> ---> 72c50c12788d</span><br><span class="line">Successfully built 72c50c12788d</span><br></pre></td></tr></table></figure><h3 id="change-image-name"><a href="#change-image-name" class="headerlink" title="change image name"></a>change image name</h3><p>I am not satisfied with my new image name, so I rename it.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/rpi# docker tag vmnet8/rpi vmnet8/base-nginx:rpi</span><br><span class="line">root@vmnet8cloud:~/rpi# docker tag vmnet8/planet vmnet8/base-nginx:x86</span><br><span class="line">root@vmnet8cloud:~/rpi# docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/base-nginx rpi 758ef3313bac 9 hours ago 52MB</span><br><span class="line">vmnet8/rpi latest 758ef3313bac 9 hours ago 52MB</span><br><span class="line">vmnet8/planet latest 72c50c12788d 9 hours ago 20.4MB</span><br><span class="line">treehouses/bettercap first fa1faf86b2e9 5 days ago 35.9MB</span><br></pre></td></tr></table></figure><h3 id="push-image-to-my-dockerhub-repo"><a href="#push-image-to-my-dockerhub-repo" class="headerlink" title="push image to my dockerhub repo"></a>push image to my dockerhub repo</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/rpi# docker push vmnet8/base-nginx:rpi</span><br><span class="line">The push refers to repository [docker.io/vmnet8/base-nginx]</span><br><span class="line">06f60bd1a60b: Mounted from vmnet8/rpi </span><br><span class="line">2d7044011d61: Mounted from vmnet8/rpi </span><br><span class="line">7772294ad95b: Mounted from vmnet8/rpi </span><br><span class="line">ca20d2ade963: Mounted from vmnet8/rpi </span><br><span class="line">0c8b16ac3ec0: Mounted from vmnet8/rpi </span><br><span class="line">6ff0e3daf858: Mounted from vmnet8/rpi </span><br><span class="line">4bc006337f97: Mounted from vmnet8/rpi </span><br><span class="line">9c66fd3856dc: Mounted from vmnet8/rpi </span><br><span class="line">77ffeb9e9842: Mounted from vmnet8/rpi </span><br><span class="line">rpi: digest: sha256:ced17c99328babcabff78c78db75c3426c47a55aa1fec8bcdb554134327f7fff size: 2205</span><br><span class="line">root@vmnet8cloud:~/rpi# docker push vmnet8/base-nginx:x86</span><br><span class="line">The push refers to repository [docker.io/vmnet8/base-nginx]</span><br><span class="line">dc696a4b831b: Mounted from vmnet8/planet </span><br><span class="line">05652609bfdb: Mounted from vmnet8/planet </span><br><span class="line">1cd1ec9f681c: Mounted from vmnet8/planet </span><br><span class="line">77ffeb9e9842: Layer already exists </span><br><span class="line">x86: digest: sha256:40fc08ff08e250a9022242cab5f68042c98b6e1f09551fe4dfca3b04dfcc8533 size: 1157</span><br></pre></td></tr></table></figure><h2 id="using-manifest-to-create-multi-arch-image"><a href="#using-manifest-to-create-multi-arch-image" class="headerlink" title="using manifest to create multi-arch image"></a>using manifest to create multi-arch image</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/rpi# docker manifest create vmnet8/base-nginx:latest vmnet8/base-nginx:x86 vmnet8/base-nginx:rpi</span><br><span class="line">Created manifest list docker.io/vmnet8/base-nginx:latest</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">root@vmnet8cloud:~/rpi# docker manifest inspect vmnet8/base-nginx</span><br><span class="line">{</span><br><span class="line"> "schemaVersion": 2,</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",</span><br><span class="line"> "manifests": [</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 2205,</span><br><span class="line"> "digest": "sha256:ced17c99328babcabff78c78db75c3426c47a55aa1fec8bcdb554134327f7fff",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "amd64",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 1157,</span><br><span class="line"> "digest": "sha256:40fc08ff08e250a9022242cab5f68042c98b6e1f09551fe4dfca3b04dfcc8533",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "arm",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> ]</span><br><span class="line">}</span><br><span class="line">root@vmnet8cloud:~/rpi# docker manifest inspect vmnet8/base-nginx</span><br><span class="line">{</span><br><span class="line"> "schemaVersion": 2,</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",</span><br><span class="line"> "manifests": [</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 2205,</span><br><span class="line"> "digest": "sha256:ced17c99328babcabff78c78db75c3426c47a55aa1fec8bcdb554134327f7fff",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "amd64",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 1157,</span><br><span class="line"> "digest": "sha256:40fc08ff08e250a9022242cab5f68042c98b6e1f09551fe4dfca3b04dfcc8533",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "arm",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> ]</span><br><span class="line">}</span><br><span class="line">root@vmnet8cloud:~/template# docker manifest push vmnet8/base-nginx:latest</span><br><span class="line">sha256:9d646d372f76466948ee660cecd0100bc56dc0fe8e73e8e5294d7873ed09519c</span><br></pre></td></tr></table></figure><h2 id="Testing"><a href="#Testing" class="headerlink" title="Testing"></a>Testing</h2><p>I created an arm arch image and amd64 image. I want to test these images to see if they work well</p><h3 id="On-my-rpi-I-test-multi-arch-if-it-works"><a href="#On-my-rpi-I-test-multi-arch-if-it-works" class="headerlink" title="On my rpi, I test multi-arch if it works"></a>On my rpi, I test multi-arch if it works</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br></pre></td><td class="code"><pre><span class="line">pi@treehouses:~ $ docker pull vmnet8/base-nginx</span><br><span class="line">Using default tag: latest</span><br><span class="line">latest: Pulling from vmnet8/base-nginx</span><br><span class="line">6e39823df636: Already exists </span><br><span class="line">44e31f35ea82: Pull complete </span><br><span class="line">42d1b29a4b84: Pull complete </span><br><span class="line">5f669d9949cf: Pull complete </span><br><span class="line">Digest: sha256:9d646d372f76466948ee660cecd0100bc56dc0fe8e73e8e5294d7873ed09519c</span><br><span class="line">Status: Downloaded newer image for vmnet8/base-nginx:latest</span><br><span class="line">docker.io/vmnet8/base-nginx:latest</span><br><span class="line">pi@treehouses:~ $ docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/base-nginx latest 72c50c12788d 12 hours ago 20.4MB</span><br><span class="line">pi@treehouses:~ $ docker inspect 72c50c12788d</span><br><span class="line">[</span><br><span class="line"> {</span><br><span class="line"> "Id": "sha256:72c50c12788d9f57b40a19b83e95a9ca1d5ede10508a6b4d9db89cac67488046",</span><br><span class="line"> "RepoTags": [</span><br><span class="line"> "vmnet8/base-nginx:latest"</span><br><span class="line"> ],</span><br><span class="line"> "RepoDigests": [</span><br><span class="line"> "vmnet8/base-nginx@sha256:9d646d372f76466948ee660cecd0100bc56dc0fe8e73e8e5294d7873ed09519c"</span><br><span class="line"> ],</span><br><span class="line"> "Parent": "",</span><br><span class="line"> "Comment": "",</span><br><span class="line"> "Created": "2019-12-09T20:16:55.461030258Z",</span><br><span class="line"> "Container": "c6f35ed83a7ef7abb9b25b5e437156de03d18a2a1d300a2cb87e73a096ec5307",</span><br><span class="line"> "ContainerConfig": {</span><br><span class="line"> "Hostname": "",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "NGINX_VERSION=1.16.0",</span><br><span class="line"> "NJS_VERSION=0.3.1",</span><br><span class="line"> "PKG_RELEASE=1"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "/bin/sh",</span><br><span class="line"> "-c",</span><br><span class="line"> "apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r1 libssh2=1.9.0-r1 libcurl=7.64.0-r3 curl=7.64.0-r3 oniguruma=6.9.4-r0 jq=1.6-r0"</span><br><span class="line"> ],</span><br><span class="line"> "Image": "sha256:f7441287619ec6a39d018a9c8a7612c719ef7f214c59a701c86734bc88d03865",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": null,</span><br><span class="line"> "OnBuild": null,</span><br><span class="line"> "Labels": {</span><br><span class="line"> "maintainer": "NGINX Docker Maintainers <[email protected]>"</span><br><span class="line"> },</span><br><span class="line"> "StopSignal": "SIGTERM"</span><br><span class="line"> },</span><br><span class="line"> "DockerVersion": "19.03.5",</span><br><span class="line"> "Author": "",</span><br><span class="line"> "Config": {</span><br><span class="line"> "Hostname": "",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "NGINX_VERSION=1.16.0",</span><br><span class="line"> "NJS_VERSION=0.3.1",</span><br><span class="line"> "PKG_RELEASE=1"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "nginx",</span><br><span class="line"> "-g",</span><br><span class="line"> "daemon off;"</span><br><span class="line"> ],</span><br><span class="line"> "ArgsEscaped": true,</span><br><span class="line"> "Image": "sha256:f7441287619ec6a39d018a9c8a7612c719ef7f214c59a701c86734bc88d03865",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": null,</span><br><span class="line"> "OnBuild": null,</span><br><span class="line"> "Labels": {</span><br><span class="line"> "maintainer": "NGINX Docker Maintainers <[email protected]>"</span><br><span class="line"> },</span><br><span class="line"> "StopSignal": "SIGTERM"</span><br><span class="line"> },</span><br><span class="line"> "Architecture": "arm",</span><br><span class="line"> "Os": "linux",</span><br><span class="line"> "Size": 20383390,</span><br><span class="line"> "VirtualSize": 20383390,</span><br><span class="line"> "GraphDriver": {</span><br><span class="line"> "Data": {</span><br><span class="line"> "LowerDir": "/var/lib/docker/overlay2/060e14f6a1d2061d067463c07859a8d771d0e062bb830765109873f70cb6069a/diff:/var/lib/docker/overlay2/30dbadfa6d758ac43c176e04ed6ebd5a5b1774068e098823c37457904cf07f9d/diff:/var/lib/docker/overlay2/decf25251786958f57e943f099fa7095fa83a5ddbca3c89130b19373a464748f/diff",</span><br><span class="line"> "MergedDir": "/var/lib/docker/overlay2/58b35d0db3363fcd6dda91aef93c198be97e0f0f1cdfe32be0077b714b1e2968/merged",</span><br><span class="line"> "UpperDir": "/var/lib/docker/overlay2/58b35d0db3363fcd6dda91aef93c198be97e0f0f1cdfe32be0077b714b1e2968/diff",</span><br><span class="line"> "WorkDir": "/var/lib/docker/overlay2/58b35d0db3363fcd6dda91aef93c198be97e0f0f1cdfe32be0077b714b1e2968/work"</span><br><span class="line"> },</span><br><span class="line"> "Name": "overlay2"</span><br><span class="line"> },</span><br><span class="line"> "RootFS": {</span><br><span class="line"> "Type": "layers",</span><br><span class="line"> "Layers": [</span><br><span class="line"> "sha256:77ffeb9e98427bfbcc3448a22e354cf57e30de34d6c516365659b851f679b1b4",</span><br><span class="line"> "sha256:1cd1ec9f681cc950d0100535cbd8f3e939c50ab9f9b2d14092992c5465a9f1da",</span><br><span class="line"> "sha256:05652609bfdb21400e2e13d494475a2e79f7427288c60d20db6756f14de59f5d",</span><br><span class="line"> "sha256:dc696a4b831ba0c0096808ba0fc072fe5927fe3b2f4189af12adf436be364f79"</span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> "Metadata": {</span><br><span class="line"> "LastTagTime": "0001-01-01T00:00:00Z"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>it shows pull arm archtecture from dockhub.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">pi@treehouses:~ $ docker run -d -p 20001:80 vmnet8/base-nginx </span><br><span class="line">8016c0ea58895053be07bc13c50a08dbe85cd01524eca608855e80b2bcffddbf</span><br></pre></td></tr></table></figure><img src="/2019/12/09/Build-multi-arch-docker-image/403.png" class=""><h3 id="try-other-image"><a href="#try-other-image" class="headerlink" title="try other image"></a>try other image</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line">pi@treehouses:~ $ docker pull vmnet8/base-nginx:rpi</span><br><span class="line">rpi: Pulling from vmnet8/base-nginx</span><br><span class="line">6e39823df636: Already exists </span><br><span class="line">38dd00ade3c1: Already exists </span><br><span class="line">c02057c17ddc: Already exists </span><br><span class="line">e6d776bad3ac: Already exists </span><br><span class="line">92ca2fd255df: Already exists </span><br><span class="line">a2cfa1cf1cbb: Already exists </span><br><span class="line">35e3be655ed6: Already exists </span><br><span class="line">f182036f54c2: Already exists </span><br><span class="line">ebbf6799aa99: Pull complete </span><br><span class="line">Digest: sha256:ced17c99328babcabff78c78db75c3426c47a55aa1fec8bcdb554134327f7fff</span><br><span class="line">Status: Downloaded newer image for vmnet8/base-nginx:rpi</span><br><span class="line">docker.io/vmnet8/base-nginx:rpi</span><br><span class="line">pi@treehouses:~ $ docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/base-nginx rpi 758ef3313bac 12 hours ago 52MB</span><br><span class="line">vmnet8/base-nginx latest 72c50c12788d 12 hours ago 20.4MB</span><br><span class="line">pi@treehouses:~ $ docker run -d -p 20002:80 vmnet8/base-nginx:rpi</span><br><span class="line">8d041aecd3937332821e569e74b6744b8d1bd63f2c3b99ea7e410819b81fec0c</span><br></pre></td></tr></table></figure><p>run this image<br>it shows 403 error</p><img src="/2019/12/09/Build-multi-arch-docker-image/404.png" class=""><h3 id="test-on-vagrant-treehouse-cli"><a href="#test-on-vagrant-treehouse-cli" class="headerlink" title="test on vagrant treehouse/cli"></a>test on vagrant treehouse/cli</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~$ docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">rabbitmq 3-management 8bdbe10dc73e 5 weeks ago 180MB</span><br><span class="line">treehouses/planet latest a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet local a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet db-init fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/planet db-init-local fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/couchdb 2.3.1 7f64c92d91fb 3 months ago 201MB</span><br><span class="line">vagrant@cli:~$ docker pull vmnet8/base-nginx</span><br><span class="line">Using default tag: latest</span><br><span class="line">latest: Pulling from vmnet8/base-nginx</span><br><span class="line">6e39823df636: Pull complete </span><br><span class="line">38dd00ade3c1: Pull complete </span><br><span class="line">c02057c17ddc: Pull complete </span><br><span class="line">e6d776bad3ac: Pull complete </span><br><span class="line">92ca2fd255df: Pull complete </span><br><span class="line">a2cfa1cf1cbb: Pull complete </span><br><span class="line">35e3be655ed6: Pull complete </span><br><span class="line">f182036f54c2: Pull complete </span><br><span class="line">ebbf6799aa99: Pull complete </span><br><span class="line">Digest: sha256:9d646d372f76466948ee660cecd0100bc56dc0fe8e73e8e5294d7873ed09519c</span><br><span class="line">Status: Downloaded newer image for vmnet8/base-nginx:latest</span><br><span class="line">docker.io/vmnet8/base-nginx:latest</span><br><span class="line">vagrant@cli:~$ docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/base-nginx latest 758ef3313bac 28 hours ago 52MB</span><br><span class="line">rabbitmq 3-management 8bdbe10dc73e 5 weeks ago 180MB</span><br><span class="line">treehouses/planet latest a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet local a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet db-init fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/planet db-init-local fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/couchdb 2.3.1 7f64c92d91fb 3 months ago 201MB</span><br><span class="line">vagrant@cli:~$ docker image inspect vmnet8/base-nginx</span><br><span class="line">[</span><br><span class="line"> {</span><br><span class="line"> "Id": "sha256:758ef3313bac7fa1fda2b08abc68fa712ae778fdaa65ef5f561e75f346e921f3",</span><br><span class="line"> "RepoTags": [</span><br><span class="line"> "vmnet8/base-nginx:latest"</span><br><span class="line"> ],</span><br><span class="line"> "RepoDigests": [</span><br><span class="line"> "vmnet8/base-nginx@sha256:9d646d372f76466948ee660cecd0100bc56dc0fe8e73e8e5294d7873ed09519c"</span><br><span class="line"> ],</span><br><span class="line"> "Parent": "",</span><br><span class="line"> "Comment": "",</span><br><span class="line"> "Created": "2019-12-09T20:30:58.028636649Z",</span><br><span class="line"> "Container": "a6902c745dd55b102fbc0fec29545b88b0b5418b81a3aa463fe6892894bae52a",</span><br><span class="line"> "ContainerConfig": {</span><br><span class="line"> "Hostname": "bf6e13d66684",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "443/tcp": {},</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "UDEV=off",</span><br><span class="line"> "NGINX_VERSION=1.14"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "/bin/sh",</span><br><span class="line"> "-c",</span><br><span class="line"> "apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r0 libssh2=1.8.2-r0 libcurl=7.64.0-r2 curl=7.64.0-r2 oniguruma=6.9.4-r0 jq=1.6-r0"</span><br><span class="line"> ],</span><br><span class="line"> "Image": "sha256:9412dc8b27c29e6114610c6abd884bb8d13b0396f800c9fed39eef921483e78c",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": null,</span><br><span class="line"> "OnBuild": [],</span><br><span class="line"> "Labels": {</span><br><span class="line"> "io.balena.architecture": "rpi",</span><br><span class="line"> "io.balena.device-type": "raspberry-pi",</span><br><span class="line"> "io.balena.qemu.version": "4.0.0+balena-arm",</span><br><span class="line"> "org.opencontainers.image.authors": "Tobias Hargesheimer <[email protected]>",</span><br><span class="line"> "org.opencontainers.image.description": "AlpineLinux with NGINX on arm arch",</span><br><span class="line"> "org.opencontainers.image.licenses": "Apache-2.0",</span><br><span class="line"> "org.opencontainers.image.source": "https://github.com/Tob1asDocker/rpi-nginx",</span><br><span class="line"> "org.opencontainers.image.title": "NGINX",</span><br><span class="line"> "org.opencontainers.image.url": "https://hub.docker.com/r/tobi312/rpi-nginx/"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> "DockerVersion": "19.03.5",</span><br><span class="line"> "Author": "",</span><br><span class="line"> "Config": {</span><br><span class="line"> "Hostname": "bf6e13d66684",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "443/tcp": {},</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "UDEV=off",</span><br><span class="line"> "NGINX_VERSION=1.14"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "nginx",</span><br><span class="line"> "-g",</span><br><span class="line"> "daemon off;"</span><br><span class="line"> ],</span><br><span class="line"> "ArgsEscaped": true,</span><br><span class="line"> "Image": "sha256:9412dc8b27c29e6114610c6abd884bb8d13b0396f800c9fed39eef921483e78c",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": [</span><br><span class="line"> "/usr/bin/entry.sh"</span><br><span class="line"> ],</span><br><span class="line"> "OnBuild": [],</span><br><span class="line"> "Labels": {</span><br><span class="line"> "io.balena.architecture": "rpi",</span><br><span class="line"> "io.balena.device-type": "raspberry-pi",</span><br><span class="line"> "io.balena.qemu.version": "4.0.0+balena-arm",</span><br><span class="line"> "org.opencontainers.image.authors": "Tobias Hargesheimer <[email protected]>",</span><br><span class="line"> "org.opencontainers.image.description": "AlpineLinux with NGINX on arm arch",</span><br><span class="line"> "org.opencontainers.image.licenses": "Apache-2.0",</span><br><span class="line"> "org.opencontainers.image.source": "https://github.com/Tob1asDocker/rpi-nginx",</span><br><span class="line"> "org.opencontainers.image.title": "NGINX",</span><br><span class="line"> "org.opencontainers.image.url": "https://hub.docker.com/r/tobi312/rpi-nginx/"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> "Architecture": "amd64",</span><br><span class="line"> "Os": "linux",</span><br><span class="line"> "Size": 52043162,</span><br><span class="line"> "VirtualSize": 52043162,</span><br><span class="line"> "GraphDriver": {</span><br><span class="line"> "Data": {</span><br><span class="line"> "LowerDir": "/var/lib/docker/overlay2/d65700181956c296d4fdad06b0eaf044450f42948c12ffe0c98369399f3a6171/diff:/var/lib/docker/overlay2/a7fb82cc01d1930806d1c39d1063f4afe95b85bd14acaf4ba5ef25e375acfcaa/diff:/var/lib/docker/overlay2/be79231a2717e7b566eb93f73e098c923310d519ba0df738e9b129e49d68309d/diff:/var/lib/docker/overlay2/85a0df8e1aee0b5a820eaea35ad4478decb1f612f76f89a58e811eb1c8cb45c5/diff:/var/lib/docker/overlay2/74bd5ba7975f879d629da5c5be5ef941598dfacc6bb696d9ac9b1e718bf43460/diff:/var/lib/docker/overlay2/723a132236875c0bbead39340a897bca76a1958e0da321bd4d5933fad0aacd92/diff:/var/lib/docker/overlay2/c6647e80ba872b63745bd7134df3ad5bd563c9723377d1d9db60bb42e14cd35c/diff:/var/lib/docker/overlay2/1bee8c0cb8bf6ace619c5c54016a0f332bf83830e3f12e401f52f88e058a7be9/diff",</span><br><span class="line"> "MergedDir": "/var/lib/docker/overlay2/b18c2588c00f54836210ae958ad1f10685db2304eeb97f501dbf9aeb9545084f/merged",</span><br><span class="line"> "UpperDir": "/var/lib/docker/overlay2/b18c2588c00f54836210ae958ad1f10685db2304eeb97f501dbf9aeb9545084f/diff",</span><br><span class="line"> "WorkDir": "/var/lib/docker/overlay2/b18c2588c00f54836210ae958ad1f10685db2304eeb97f501dbf9aeb9545084f/work"</span><br><span class="line"> },</span><br><span class="line"> "Name": "overlay2"</span><br><span class="line"> },</span><br><span class="line"> "RootFS": {</span><br><span class="line"> "Type": "layers",</span><br><span class="line"> "Layers": [</span><br><span class="line"> "sha256:77ffeb9e98427bfbcc3448a22e354cf57e30de34d6c516365659b851f679b1b4",</span><br><span class="line"> "sha256:9c66fd3856dc997fc3aedba30803f1bd69651be0cda1757d5db959f37a7fe491",</span><br><span class="line"> "sha256:4bc006337f970a3226b3ab7b5d3692b90d9bb7cd18746c29be269d6ed75f347d",</span><br><span class="line"> "sha256:6ff0e3daf8589cf6b0e6f254f3f7d44d0955810d147bd4f87b150d794d6aa681",</span><br><span class="line"> "sha256:0c8b16ac3ec05312e8fc393f53fe14840a0c8408c960f75297b46ad252f17d8a",</span><br><span class="line"> "sha256:ca20d2ade963c426acdb97ca81df2d1b0181fc29f1b89745d582874e64c9087f",</span><br><span class="line"> "sha256:7772294ad95bdaa10c84d306f03bbcc74bede2cc04393114f7c821c0813a8828",</span><br><span class="line"> "sha256:2d7044011d61cb9fa0a42c0741341323240bb6c1ccb4741afacbb0bbdaaa3dcb",</span><br><span class="line"> "sha256:06f60bd1a60b1ebeebb9d569809e0282ff9ccb7a1ea235c35e66e618dd993b94"</span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> "Metadata": {</span><br><span class="line"> "LastTagTime": "0001-01-01T00:00:00Z"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>docker pull the amd64 as latest version.</p><p>run this image</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~$ docker run -d -p 20003:80 vmnet8/base-nginx</span><br><span class="line">3a0dd81e2b75c5cf34d33cd41308f66049b072f01543bc079bfaed6246d20cfd</span><br><span class="line">vagrant@cli:~$ curl localhost:20003</span><br><span class="line">curl: (7) Failed to connect to localhost port 20003: Connection refused</span><br><span class="line">vagrant@cli:~$ docker ps -a</span><br><span class="line">CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES</span><br><span class="line">3a0dd81e2b75 vmnet8/base-nginx "/usr/bin/entry.sh n…" 17 seconds ago Exited (1) 15 seconds ago confident_poincare</span><br><span class="line">cb31b79eff78 vmnet8/base-nginx "/usr/bin/entry.sh n…" 5 hours ago Exited (1) 5 hours ago stoic_williamson</span><br><span class="line">a84339e656b4 rabbitmq:3-management "docker-entrypoint.s…" 5 weeks ago Exited (255) 5 hours ago 4369/tcp, 5671-5672/tcp, 15671/tcp, 25672/tcp, 0.0.0.0:8080->15672/tcp some-rabbit</span><br><span class="line">6b2ff2ae5528 treehouses/couchdb:2.3.1 "tini -- /docker-ent…" 6 weeks ago Exited (255) 5 hours ago 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp vmnet8</span><br></pre></td></tr></table></figure><p>it shows this docker image failed to run. I build this image on Pi, which is the reason failed.</p><h3 id="build-image-on-x86"><a href="#build-image-on-x86" class="headerlink" title="build image on x86."></a>build image on x86.</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/newbase$ docker image build -t vmnet8/base-nginx:x86new .</span><br><span class="line">Sending build context to Docker daemon 2.048kB</span><br><span class="line">Step 1/3 : FROM nginx:1.16.0-alpine</span><br><span class="line">1.16.0-alpine: Pulling from library/nginx</span><br><span class="line">e7c96db7181b: Already exists </span><br><span class="line">8985e402e050: Already exists </span><br><span class="line">Digest: sha256:270bea203d2fc3743fb9ce0193325e188b7e6233043487e3d3cf117ea4d3f337</span><br><span class="line">Status: Downloaded newer image for nginx:1.16.0-alpine</span><br><span class="line"> ---> ef04b00b089d</span><br><span class="line">Step 2/3 : RUN rm -rf /usr/share/nginx/html/*</span><br><span class="line"> ---> Running in 26a738b05d6b</span><br><span class="line">Removing intermediate container 26a738b05d6b</span><br><span class="line"> ---> 1a3fa7c937d7</span><br><span class="line">Step 3/3 : RUN apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r1 libssh2=1.9.0-r1 libcurl=7.64.0-r3 curl=7.64.0-r3 oniguruma=6.9.4-r0 jq=1.6-r0</span><br><span class="line"> ---> Running in 487a73601a7c</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/main/x86_64/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.9/community/x86_64/APKINDEX.tar.gz</span><br><span class="line">(1/10) Installing ca-certificates (20190108-r0)</span><br><span class="line">(2/10) Installing nghttp2-libs (1.35.1-r1)</span><br><span class="line">(3/10) Installing libssh2 (1.9.0-r1)</span><br><span class="line">(4/10) Installing libcurl (7.64.0-r3)</span><br><span class="line">(5/10) Installing curl (7.64.0-r3)</span><br><span class="line">(6/10) Installing fcgi (2.4.0-r8)</span><br><span class="line">(7/10) Installing fcgiwrap (1.1.0-r3)</span><br><span class="line">Executing fcgiwrap-1.1.0-r3.pre-install</span><br><span class="line">(8/10) Installing oniguruma (6.9.4-r0)</span><br><span class="line">(9/10) Installing jq (1.6-r0)</span><br><span class="line">(10/10) Installing spawn-fcgi (1.6.4-r3)</span><br><span class="line">Executing busybox-1.29.3-r10.trigger</span><br><span class="line">Executing ca-certificates-20190108-r0.trigger</span><br><span class="line">OK: 30 MiB in 47 packages</span><br><span class="line">Removing intermediate container 487a73601a7c</span><br><span class="line"> ---> d8aec7e71b73</span><br><span class="line">Successfully built d8aec7e71b73</span><br><span class="line">Successfully tagged vmnet8/base-nginx:x86new</span><br><span class="line">vagrant@cli:~/newbase$ docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/base-nginx x86new d8aec7e71b73 14 seconds ago 23.5MB</span><br><span class="line">vmnet8/base-nginx latest 758ef3313bac 33 hours ago 52MB</span><br><span class="line">rabbitmq 3-management 8bdbe10dc73e 5 weeks ago 180MB</span><br><span class="line">treehouses/planet latest a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet local a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet db-init fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/planet db-init-local fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/couchdb 2.3.1 7f64c92d91fb 3 months ago 201MB</span><br><span class="line">nginx 1.16.0-alpine ef04b00b089d 7 months ago 20.4MB</span><br><span class="line">vagrant@cli:~/newbase$ docker image inspect vmnet8/base-nginx:x86new</span><br><span class="line">[</span><br><span class="line"> {</span><br><span class="line"> "Id": "sha256:d8aec7e71b735ce78ea6e30fa22272c29206d064302b04b5265ba7abfa60250a",</span><br><span class="line"> "RepoTags": [</span><br><span class="line"> "vmnet8/base-nginx:x86new"</span><br><span class="line"> ],</span><br><span class="line"> "RepoDigests": [],</span><br><span class="line"> "Parent": "sha256:1a3fa7c937d7f6dfa6e1353bc404f2bdecbc650de6ee68274e512dbf98698d5c",</span><br><span class="line"> "Comment": "",</span><br><span class="line"> "Created": "2019-12-11T05:41:23.768015601Z",</span><br><span class="line"> "Container": "487a73601a7c8890adb98c479a2b711e04b5d148b3bc8d90e5ee741034a08971",</span><br><span class="line"> "ContainerConfig": {</span><br><span class="line"> "Hostname": "",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "NGINX_VERSION=1.16.0",</span><br><span class="line"> "NJS_VERSION=0.3.1",</span><br><span class="line"> "PKG_RELEASE=1"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "/bin/sh",</span><br><span class="line"> "-c",</span><br><span class="line"> "apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r1 libssh2=1.9.0-r1 libcurl=7.64.0-r3 curl=7.64.0-r3 oniguruma=6.9.4-r0 jq=1.6-r0"</span><br><span class="line"> ],</span><br><span class="line"> "Image": "sha256:1a3fa7c937d7f6dfa6e1353bc404f2bdecbc650de6ee68274e512dbf98698d5c",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": null,</span><br><span class="line"> "OnBuild": null,</span><br><span class="line"> "Labels": {</span><br><span class="line"> "maintainer": "NGINX Docker Maintainers <[email protected]>"</span><br><span class="line"> },</span><br><span class="line"> "StopSignal": "SIGTERM"</span><br><span class="line"> },</span><br><span class="line"> "DockerVersion": "19.03.3",</span><br><span class="line"> "Author": "",</span><br><span class="line"> "Config": {</span><br><span class="line"> "Hostname": "",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "NGINX_VERSION=1.16.0",</span><br><span class="line"> "NJS_VERSION=0.3.1",</span><br><span class="line"> "PKG_RELEASE=1"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "nginx",</span><br><span class="line"> "-g",</span><br><span class="line"> "daemon off;"</span><br><span class="line"> ],</span><br><span class="line"> "ArgsEscaped": true,</span><br><span class="line"> "Image": "sha256:1a3fa7c937d7f6dfa6e1353bc404f2bdecbc650de6ee68274e512dbf98698d5c",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": null,</span><br><span class="line"> "OnBuild": null,</span><br><span class="line"> "Labels": {</span><br><span class="line"> "maintainer": "NGINX Docker Maintainers <[email protected]>"</span><br><span class="line"> },</span><br><span class="line"> "StopSignal": "SIGTERM"</span><br><span class="line"> },</span><br><span class="line"> "Architecture": "amd64",</span><br><span class="line"> "Os": "linux",</span><br><span class="line"> "Size": 23454243,</span><br><span class="line"> "VirtualSize": 23454243,</span><br><span class="line"> "GraphDriver": {</span><br><span class="line"> "Data": {</span><br><span class="line"> "LowerDir": "/var/lib/docker/overlay2/5a707ebb56c473f0b857d9f3ba70210ec9e599e2c1dcff32091ff9609e97fc40/diff:/var/lib/docker/overlay2/dc6895345b17265c1e0d24dfd5e54755dd564c41f31e8e43cc53ecfa559aabdf/diff:/var/lib/docker/overlay2/75e6526fd72d6193688bb1d4f9b3eea897e2e3fc0467f1ff39e4ef94e4a9ee61/diff",</span><br><span class="line"> "MergedDir": "/var/lib/docker/overlay2/4c4adc9af1f46fcb4bcf1b0d90715b31302dc6b1861d34ea85191d3bd59b5fa9/merged",</span><br><span class="line"> "UpperDir": "/var/lib/docker/overlay2/4c4adc9af1f46fcb4bcf1b0d90715b31302dc6b1861d34ea85191d3bd59b5fa9/diff",</span><br><span class="line"> "WorkDir": "/var/lib/docker/overlay2/4c4adc9af1f46fcb4bcf1b0d90715b31302dc6b1861d34ea85191d3bd59b5fa9/work"</span><br><span class="line"> },</span><br><span class="line"> "Name": "overlay2"</span><br><span class="line"> },</span><br><span class="line"> "RootFS": {</span><br><span class="line"> "Type": "layers",</span><br><span class="line"> "Layers": [</span><br><span class="line"> "sha256:f1b5933fe4b5f49bbe8258745cf396afe07e625bdab3168e364daf7c956b6b81",</span><br><span class="line"> "sha256:2bdf88b2699da3549a9cbc76f9b6ee058cc1f9d232fba48dda107478abfc694b",</span><br><span class="line"> "sha256:cc2d7d10b40de1e2253f628b1881b4e27998b82ba38f8182f4773144a30c7ca0",</span><br><span class="line"> "sha256:6f9350f71e829ee3a008387295156dd216e126c2248b0ce9e72a6dfdb599a6ac"</span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> "Metadata": {</span><br><span class="line"> "LastTagTime": "2019-12-11T05:41:23.967401201Z"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>I pushed this image to dockerhub</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/newbase$ docker push vmnet8/base-nginx:x86new</span><br><span class="line">The push refers to repository [docker.io/vmnet8/base-nginx]</span><br><span class="line">6f9350f71e82: Pushed </span><br><span class="line">cc2d7d10b40d: Pushed </span><br><span class="line">2bdf88b2699d: Mounted from library/nginx </span><br><span class="line">f1b5933fe4b5: Mounted from library/nginx </span><br><span class="line">x86new: digest: sha256:ec6b9bd2bdac454ab43080c163ba9e06119e0a075c5f433cbcd481398c88a013 size: 1157</span><br></pre></td></tr></table></figure><p>I try to build multi-arch image</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/newbase$ docker manifest create vmnet8/base-nginx:latest vmnet8/base-nginx:rpi vmnet8/base-nginx:x86 vmnet8/base-nginx:x86new </span><br><span class="line">Created manifest list docker.io/vmnet8/base-nginx:latest</span><br></pre></td></tr></table></figure><p>note: before manifest create, I need to </p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim /home/vagrant/.docker/config.json</span><br></pre></td></tr></table></figure><p>add </p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">"experimental": "enabled"</span><br></pre></td></tr></table></figure><p>after manifest created, I inspect it</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/newbase$ docker manifest inspect vmnet8/base-nginx</span><br><span class="line">{</span><br><span class="line"> "schemaVersion": 2,</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",</span><br><span class="line"> "manifests": [</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 2205,</span><br><span class="line"> "digest": "sha256:ced17c99328babcabff78c78db75c3426c47a55aa1fec8bcdb554134327f7fff",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "amd64",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 1157,</span><br><span class="line"> "digest": "sha256:40fc08ff08e250a9022242cab5f68042c98b6e1f09551fe4dfca3b04dfcc8533",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "arm",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> {</span><br><span class="line"> "mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line"> "size": 1157,</span><br><span class="line"> "digest": "sha256:ec6b9bd2bdac454ab43080c163ba9e06119e0a075c5f433cbcd481398c88a013",</span><br><span class="line"> "platform": {</span><br><span class="line"> "architecture": "amd64",</span><br><span class="line"> "os": "linux"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line"> ]</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>then push this manifest list to dockerhub</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/newbase$ docker manifest push vmnet8/base-nginx:latest</span><br><span class="line">sha256:49c78057a124c4b05f90028d3d322c39e4f4b76ee2be173e6969207f7887cc77</span><br></pre></td></tr></table></figure><p>I am going to run imaged which is created on vagrant to see if it works</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/newbase$ docker run -d -p 20003:80 vmnet8/base-nginx:x86new</span><br><span class="line">9ab8b587f00d2d5df112a12acff0f1bb43ec5a99880c20566d05be83bfb81eaa</span><br><span class="line">vagrant@cli:~/newbase$ curl localhost:20003</span><br><span class="line"><html></span><br><span class="line"><head><title>403 Forbidden</title></head></span><br><span class="line"><body></span><br><span class="line"><center><h1>403 Forbidden</h1></center></span><br><span class="line"><hr><center>nginx/1.16.0</center></span><br><span class="line"></body></span><br><span class="line"></html></span><br></pre></td></tr></table></figure><p>it works!</p><p>I try to use multi-arch tag to pull docker image from dockerhub</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~$ docker pull vmnet8/base-nginx</span><br><span class="line">Using default tag: latest</span><br><span class="line">latest: Pulling from vmnet8/base-nginx</span><br><span class="line">6e39823df636: Pull complete </span><br><span class="line">38dd00ade3c1: Pull complete </span><br><span class="line">c02057c17ddc: Pull complete </span><br><span class="line">e6d776bad3ac: Pull complete </span><br><span class="line">92ca2fd255df: Pull complete </span><br><span class="line">a2cfa1cf1cbb: Pull complete </span><br><span class="line">35e3be655ed6: Pull complete </span><br><span class="line">f182036f54c2: Pull complete </span><br><span class="line">ebbf6799aa99: Pull complete </span><br><span class="line">Digest: sha256:49c78057a124c4b05f90028d3d322c39e4f4b76ee2be173e6969207f7887cc77</span><br><span class="line">Status: Downloaded newer image for vmnet8/base-nginx:latest</span><br><span class="line">docker.io/vmnet8/base-nginx:latest</span><br><span class="line">vagrant@cli:~$ docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/base-nginx latest 758ef3313bac 34 hours ago 52MB</span><br><span class="line">rabbitmq 3-management 8bdbe10dc73e 5 weeks ago 180MB</span><br><span class="line">treehouses/planet latest a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet local a67c7ab86add 7 weeks ago 37.6MB</span><br><span class="line">treehouses/planet db-init fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/planet db-init-local fab533186a58 7 weeks ago 98.6MB</span><br><span class="line">treehouses/couchdb 2.3.1 7f64c92d91fb 3 months ago 201MB</span><br><span class="line">nginx 1.16.0-alpine ef04b00b089d 7 months ago 20.4MB</span><br><span class="line">vagrant@cli:~$ docker run -d -p 20004:80 vmnet8/base-nginx</span><br><span class="line">a89bcd5de4cc0f891df5a07d86d0a0424f72c307924ed814a87fb84110441a13</span><br><span class="line">vagrant@cli:~$ curl localhost:20004</span><br><span class="line">curl: (7) Failed to connect to localhost port 20004: Connection refused</span><br><span class="line">vagrant@cli:~$ docker image inspect vmnet8/base-nginx</span><br><span class="line">[</span><br><span class="line"> {</span><br><span class="line"> "Id": "sha256:758ef3313bac7fa1fda2b08abc68fa712ae778fdaa65ef5f561e75f346e921f3",</span><br><span class="line"> "RepoTags": [</span><br><span class="line"> "vmnet8/base-nginx:latest"</span><br><span class="line"> ],</span><br><span class="line"> "RepoDigests": [</span><br><span class="line"> "vmnet8/base-nginx@sha256:49c78057a124c4b05f90028d3d322c39e4f4b76ee2be173e6969207f7887cc77"</span><br><span class="line"> ],</span><br><span class="line"> "Parent": "",</span><br><span class="line"> "Comment": "",</span><br><span class="line"> "Created": "2019-12-09T20:30:58.028636649Z",</span><br><span class="line"> "Container": "a6902c745dd55b102fbc0fec29545b88b0b5418b81a3aa463fe6892894bae52a",</span><br><span class="line"> "ContainerConfig": {</span><br><span class="line"> "Hostname": "bf6e13d66684",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "443/tcp": {},</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "UDEV=off",</span><br><span class="line"> "NGINX_VERSION=1.14"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "/bin/sh",</span><br><span class="line"> "-c",</span><br><span class="line"> "apk add --no-cache fcgi=2.4.0-r8 fcgiwrap=1.1.0-r3 spawn-fcgi=1.6.4-r3 ca-certificates=20190108-r0 nghttp2-libs=1.35.1-r0 libssh2=1.8.2-r0 libcurl=7.64.0-r2 curl=7.64.0-r2 oniguruma=6.9.4-r0 jq=1.6-r0"</span><br><span class="line"> ],</span><br><span class="line"> "Image": "sha256:9412dc8b27c29e6114610c6abd884bb8d13b0396f800c9fed39eef921483e78c",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": null,</span><br><span class="line"> "OnBuild": [],</span><br><span class="line"> "Labels": {</span><br><span class="line"> "io.balena.architecture": "rpi",</span><br><span class="line"> "io.balena.device-type": "raspberry-pi",</span><br><span class="line"> "io.balena.qemu.version": "4.0.0+balena-arm",</span><br><span class="line"> "org.opencontainers.image.authors": "Tobias Hargesheimer <[email protected]>",</span><br><span class="line"> "org.opencontainers.image.description": "AlpineLinux with NGINX on arm arch",</span><br><span class="line"> "org.opencontainers.image.licenses": "Apache-2.0",</span><br><span class="line"> "org.opencontainers.image.source": "https://github.com/Tob1asDocker/rpi-nginx",</span><br><span class="line"> "org.opencontainers.image.title": "NGINX",</span><br><span class="line"> "org.opencontainers.image.url": "https://hub.docker.com/r/tobi312/rpi-nginx/"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> "DockerVersion": "19.03.5",</span><br><span class="line"> "Author": "",</span><br><span class="line"> "Config": {</span><br><span class="line"> "Hostname": "bf6e13d66684",</span><br><span class="line"> "Domainname": "",</span><br><span class="line"> "User": "",</span><br><span class="line"> "AttachStdin": false,</span><br><span class="line"> "AttachStdout": false,</span><br><span class="line"> "AttachStderr": false,</span><br><span class="line"> "ExposedPorts": {</span><br><span class="line"> "443/tcp": {},</span><br><span class="line"> "80/tcp": {}</span><br><span class="line"> },</span><br><span class="line"> "Tty": false,</span><br><span class="line"> "OpenStdin": false,</span><br><span class="line"> "StdinOnce": false,</span><br><span class="line"> "Env": [</span><br><span class="line"> "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",</span><br><span class="line"> "UDEV=off",</span><br><span class="line"> "NGINX_VERSION=1.14"</span><br><span class="line"> ],</span><br><span class="line"> "Cmd": [</span><br><span class="line"> "nginx",</span><br><span class="line"> "-g",</span><br><span class="line"> "daemon off;"</span><br><span class="line"> ],</span><br><span class="line"> "ArgsEscaped": true,</span><br><span class="line"> "Image": "sha256:9412dc8b27c29e6114610c6abd884bb8d13b0396f800c9fed39eef921483e78c",</span><br><span class="line"> "Volumes": null,</span><br><span class="line"> "WorkingDir": "",</span><br><span class="line"> "Entrypoint": [</span><br><span class="line"> "/usr/bin/entry.sh"</span><br><span class="line"> ],</span><br><span class="line"> "OnBuild": [],</span><br><span class="line"> "Labels": {</span><br><span class="line"> "io.balena.architecture": "rpi",</span><br><span class="line"> "io.balena.device-type": "raspberry-pi",</span><br><span class="line"> "io.balena.qemu.version": "4.0.0+balena-arm",</span><br><span class="line"> "org.opencontainers.image.authors": "Tobias Hargesheimer <[email protected]>",</span><br><span class="line"> "org.opencontainers.image.description": "AlpineLinux with NGINX on arm arch",</span><br><span class="line"> "org.opencontainers.image.licenses": "Apache-2.0",</span><br><span class="line"> "org.opencontainers.image.source": "https://github.com/Tob1asDocker/rpi-nginx",</span><br><span class="line"> "org.opencontainers.image.title": "NGINX",</span><br><span class="line"> "org.opencontainers.image.url": "https://hub.docker.com/r/tobi312/rpi-nginx/"</span><br><span class="line"> }</span><br><span class="line"> },</span><br><span class="line"> "Architecture": "amd64",</span><br><span class="line"> "Os": "linux",</span><br><span class="line"> "Size": 52043162,</span><br><span class="line"> "VirtualSize": 52043162,</span><br><span class="line"> "GraphDriver": {</span><br><span class="line"> "Data": {</span><br><span class="line"> "LowerDir": "/var/lib/docker/overlay2/70cc3e3fca1507e7c9211d22d66265395f885288b1d399fd003514537556386b/diff:/var/lib/docker/overlay2/ab42b343ec4b7c5aa9e0ba0f481ace9ae691ad10afdf36f1c314b230342f9924/diff:/var/lib/docker/overlay2/7e603a7ebe90bb63e84dd3501b14ba46c0616240f7c52aaa47197b9e3c804f33/diff:/var/lib/docker/overlay2/b8dd6469e761a22a9dfe53b5289a90b779b4fbdf26fcb1dd26756611d81a64ef/diff:/var/lib/docker/overlay2/3fc0c51f817104f7fca2a82f0802196eb6a57a8cf4decc787c52b99fd54f4b7c/diff:/var/lib/docker/overlay2/b89a682e4c598baceee669bf3fdefc299975da996e6d0013a5090c301aeae075/diff:/var/lib/docker/overlay2/bf6fd4d2a150b622243975d715f0e33e556071b13c6484fe7fe5917b08b71481/diff:/var/lib/docker/overlay2/fd3f10fb826ce429ce564d9df304dd67ef3ddb51ad6f8c25997d56db45705e5c/diff",</span><br><span class="line"> "MergedDir": "/var/lib/docker/overlay2/7fde1119a0359c349ce4626798034f5bfe595f5137996b86d7604e9f0547f4ee/merged",</span><br><span class="line"> "UpperDir": "/var/lib/docker/overlay2/7fde1119a0359c349ce4626798034f5bfe595f5137996b86d7604e9f0547f4ee/diff",</span><br><span class="line"> "WorkDir": "/var/lib/docker/overlay2/7fde1119a0359c349ce4626798034f5bfe595f5137996b86d7604e9f0547f4ee/work"</span><br><span class="line"> },</span><br><span class="line"> "Name": "overlay2"</span><br><span class="line"> },</span><br><span class="line"> "RootFS": {</span><br><span class="line"> "Type": "layers",</span><br><span class="line"> "Layers": [</span><br><span class="line"> "sha256:77ffeb9e98427bfbcc3448a22e354cf57e30de34d6c516365659b851f679b1b4",</span><br><span class="line"> "sha256:9c66fd3856dc997fc3aedba30803f1bd69651be0cda1757d5db959f37a7fe491",</span><br><span class="line"> "sha256:4bc006337f970a3226b3ab7b5d3692b90d9bb7cd18746c29be269d6ed75f347d",</span><br><span class="line"> "sha256:6ff0e3daf8589cf6b0e6f254f3f7d44d0955810d147bd4f87b150d794d6aa681",</span><br><span class="line"> "sha256:0c8b16ac3ec05312e8fc393f53fe14840a0c8408c960f75297b46ad252f17d8a",</span><br><span class="line"> "sha256:ca20d2ade963c426acdb97ca81df2d1b0181fc29f1b89745d582874e64c9087f",</span><br><span class="line"> "sha256:7772294ad95bdaa10c84d306f03bbcc74bede2cc04393114f7c821c0813a8828",</span><br><span class="line"> "sha256:2d7044011d61cb9fa0a42c0741341323240bb6c1ccb4741afacbb0bbdaaa3dcb",</span><br><span class="line"> "sha256:06f60bd1a60b1ebeebb9d569809e0282ff9ccb7a1ea235c35e66e618dd993b94"</span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> "Metadata": {</span><br><span class="line"> "LastTagTime": "0001-01-01T00:00:00Z"</span><br><span class="line"> }</span><br><span class="line"> }</span><br><span class="line">]</span><br></pre></td></tr></table></figure><p>it still failed, but I don’t know which tag this image is</p>]]></content>
<tags>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Build a docker image: bettercap</title>
<link href="/2019/12/06/Build-a-docker-image-bettercap/"/>
<url>/2019/12/06/Build-a-docker-image-bettercap/</url>
<content type="html"><![CDATA[<p>Make directory for bettercap and build docker image under this diretory, not recommend under root.<br>Clone the dockerfile from github,<br>Note: git clone the whole git repo, otherwise build process will failed because of lacking some files such as makefile.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git clone https://github.com/bettercap/bettercap.git</span><br></pre></td></tr></table></figure><h2 id="build-the-docker-image"><a href="#build-the-docker-image" class="headerlink" title="build the docker image"></a>build the docker image</h2><p>Note: it needs at least 4G memory. I used the Pi 3B+, which only has 1G memory, it failed. then I changed to Pi4.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br><span class="line">193</span><br><span class="line">194</span><br><span class="line">195</span><br><span class="line">196</span><br><span class="line">197</span><br><span class="line">198</span><br><span class="line">199</span><br><span class="line">200</span><br><span class="line">201</span><br><span class="line">202</span><br><span class="line">203</span><br><span class="line">204</span><br><span class="line">205</span><br><span class="line">206</span><br><span class="line">207</span><br><span class="line">208</span><br><span class="line">209</span><br><span class="line">210</span><br><span class="line">211</span><br><span class="line">212</span><br><span class="line">213</span><br><span class="line">214</span><br><span class="line">215</span><br><span class="line">216</span><br><span class="line">217</span><br><span class="line">218</span><br><span class="line">219</span><br><span class="line">220</span><br><span class="line">221</span><br><span class="line">222</span><br><span class="line">223</span><br><span class="line">224</span><br><span class="line">225</span><br><span class="line">226</span><br><span class="line">227</span><br><span class="line">228</span><br><span class="line">229</span><br><span class="line">230</span><br><span class="line">231</span><br><span class="line">232</span><br><span class="line">233</span><br><span class="line">234</span><br><span class="line">235</span><br><span class="line">236</span><br><span class="line">237</span><br><span class="line">238</span><br><span class="line">239</span><br><span class="line">240</span><br><span class="line">241</span><br><span class="line">242</span><br><span class="line">243</span><br><span class="line">244</span><br><span class="line">245</span><br><span class="line">246</span><br><span class="line">247</span><br><span class="line">248</span><br><span class="line">249</span><br><span class="line">250</span><br><span class="line">251</span><br><span class="line">252</span><br><span class="line">253</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker build .</span><br><span class="line">Sending build context to Docker daemon 19.83MB</span><br><span class="line">Step 1/17 : FROM golang:alpine AS build-env</span><br><span class="line"> ---> 6a83a6459979</span><br><span class="line">Step 2/17 : ENV SRC_DIR $GOPATH/src/github.com/bettercap/bettercap</span><br><span class="line"> ---> Running in f7be9c390ea2</span><br><span class="line">Removing intermediate container f7be9c390ea2</span><br><span class="line"> ---> 574512a28fc9</span><br><span class="line">Step 3/17 : RUN apk add --update ca-certificates</span><br><span class="line"> ---> Running in b93f2b7777b8</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/armv7/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/armv7/APKINDEX.tar.gz</span><br><span class="line">OK: 4 MiB in 15 packages</span><br><span class="line">Removing intermediate container b93f2b7777b8</span><br><span class="line"> ---> 6a02504d09c8</span><br><span class="line">Step 4/17 : RUN apk add --no-cache --update bash iptables wireless-tools build-base libpcap-dev libusb-dev linux-headers libnetfilter_queue-dev git</span><br><span class="line"> ---> Running in 504d45657d66</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/armv7/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/armv7/APKINDEX.tar.gz</span><br><span class="line">(1/42) Installing ncurses-terminfo-base (6.1_p20190518-r0)</span><br><span class="line">(2/42) Installing ncurses-terminfo (6.1_p20190518-r0)</span><br><span class="line">(3/42) Installing ncurses-libs (6.1_p20190518-r0)</span><br><span class="line">(4/42) Installing readline (8.0.0-r0)</span><br><span class="line">(5/42) Installing bash (5.0.0-r0)</span><br><span class="line">Executing bash-5.0.0-r0.post-install</span><br><span class="line">(6/42) Installing binutils (2.32-r0)</span><br><span class="line">(7/42) Installing libmagic (5.37-r1)</span><br><span class="line">(8/42) Installing file (5.37-r1)</span><br><span class="line">(9/42) Installing gmp (6.1.2-r1)</span><br><span class="line">(10/42) Installing isl (0.18-r0)</span><br><span class="line">(11/42) Installing libgomp (8.3.0-r0)</span><br><span class="line">(12/42) Installing libatomic (8.3.0-r0)</span><br><span class="line">(13/42) Installing libgcc (8.3.0-r0)</span><br><span class="line">(14/42) Installing mpfr3 (3.1.5-r1)</span><br><span class="line">(15/42) Installing mpc1 (1.1.0-r0)</span><br><span class="line">(16/42) Installing libstdc++ (8.3.0-r0)</span><br><span class="line">(17/42) Installing gcc (8.3.0-r0)</span><br><span class="line">(18/42) Installing musl-dev (1.1.22-r3)</span><br><span class="line">(19/42) Installing libc-dev (0.7.1-r0)</span><br><span class="line">(20/42) Installing g++ (8.3.0-r0)</span><br><span class="line">(21/42) Installing make (4.2.1-r2)</span><br><span class="line">(22/42) Installing fortify-headers (1.1-r0)</span><br><span class="line">(23/42) Installing build-base (0.5-r1)</span><br><span class="line">(24/42) Installing nghttp2-libs (1.39.2-r0)</span><br><span class="line">(25/42) Installing libcurl (7.66.0-r0)</span><br><span class="line">(26/42) Installing expat (2.2.8-r0)</span><br><span class="line">(27/42) Installing pcre2 (10.33-r0)</span><br><span class="line">(28/42) Installing git (2.22.0-r0)</span><br><span class="line">(29/42) Installing libmnl (1.0.4-r0)</span><br><span class="line">(30/42) Installing libnftnl-libs (1.1.3-r0)</span><br><span class="line">(31/42) Installing iptables (1.8.3-r0)</span><br><span class="line">(32/42) Installing libnfnetlink (1.0.1-r1)</span><br><span class="line">(33/42) Installing libnetfilter_queue (1.0.3-r0)</span><br><span class="line">(34/42) Installing pkgconf (1.6.1-r1)</span><br><span class="line">(35/42) Installing libnfnetlink-dev (1.0.1-r1)</span><br><span class="line">(36/42) Installing libnetfilter_queue-dev (1.0.3-r0)</span><br><span class="line">(37/42) Installing libpcap (1.9.1-r0)</span><br><span class="line">(38/42) Installing libpcap-dev (1.9.1-r0)</span><br><span class="line">(39/42) Installing libusb (1.0.22-r0)</span><br><span class="line">(40/42) Installing libusb-dev (1.0.22-r0)</span><br><span class="line">(41/42) Installing linux-headers (4.19.36-r0)</span><br><span class="line">(42/42) Installing wireless-tools (30_pre9-r1)</span><br><span class="line">Executing busybox-1.30.1-r2.trigger</span><br><span class="line">OK: 144 MiB in 57 packages</span><br><span class="line">Removing intermediate container 504d45657d66</span><br><span class="line"> ---> f648d787ff0e</span><br><span class="line">Step 5/17 : WORKDIR $SRC_DIR</span><br><span class="line"> ---> Running in e14fcfed0d87</span><br><span class="line">Removing intermediate container e14fcfed0d87</span><br><span class="line"> ---> 181d2b3b343f</span><br><span class="line">Step 6/17 : ADD . $SRC_DIR</span><br><span class="line"> ---> 4461303d94d7</span><br><span class="line">Step 7/17 : RUN make</span><br><span class="line"> ---> Running in 1faa1d85eb96</span><br><span class="line">go build -o bettercap .</span><br><span class="line">go: downloading github.com/evilsocket/islazy v1.10.4</span><br><span class="line">go: downloading github.com/dustin/go-humanize v1.0.0</span><br><span class="line">go: downloading github.com/bettercap/readline v0.0.0-20180208083827-9cec905dd291</span><br><span class="line">go: downloading github.com/google/gopacket v1.1.17</span><br><span class="line">go: downloading github.com/jpillora/go-tld v0.0.0-20190202073305-f16ca3b7b383</span><br><span class="line">go: extracting github.com/dustin/go-humanize v1.0.0</span><br><span class="line">go: downloading github.com/gobwas/glob v0.0.0-20181002190808-e7a84e9525fe</span><br><span class="line">go: extracting github.com/evilsocket/islazy v1.10.4</span><br><span class="line">go: extracting github.com/bettercap/readline v0.0.0-20180208083827-9cec905dd291</span><br><span class="line">go: downloading github.com/malfunkt/iprange v0.9.0</span><br><span class="line">go: downloading github.com/antchfx/jsonquery v1.0.0</span><br><span class="line">go: downloading github.com/miekg/dns v1.1.22</span><br><span class="line">go: downloading github.com/bettercap/nrf24 v0.0.0-20190219153547-aa37e6d0e0eb</span><br><span class="line">go: downloading github.com/google/gousb v0.0.0-20190812193832-18f4c1d8a750</span><br><span class="line">go: extracting github.com/antchfx/jsonquery v1.0.0</span><br><span class="line">go: extracting github.com/malfunkt/iprange v0.9.0</span><br><span class="line">go: extracting github.com/jpillora/go-tld v0.0.0-20190202073305-f16ca3b7b383</span><br><span class="line">go: extracting github.com/bettercap/nrf24 v0.0.0-20190219153547-aa37e6d0e0eb</span><br><span class="line">go: extracting github.com/gobwas/glob v0.0.0-20181002190808-e7a84e9525fe</span><br><span class="line">go: downloading github.com/pkg/errors v0.8.1</span><br><span class="line">go: downloading github.com/inconshreveable/go-vhost v0.0.0-20160627193104-06d84117953b</span><br><span class="line">go: downloading github.com/robertkrimen/otto v0.0.0-20180617131154-15f95af6e78d</span><br><span class="line">go: downloading github.com/antchfx/xpath v1.1.0</span><br><span class="line">go: downloading github.com/bettercap/recording v0.0.0-20190408083647-3ce1dcf032e3</span><br><span class="line">go: extracting github.com/miekg/dns v1.1.22</span><br><span class="line">go: extracting github.com/google/gousb v0.0.0-20190812193832-18f4c1d8a750</span><br><span class="line">go: extracting github.com/pkg/errors v0.8.1</span><br><span class="line">go: extracting github.com/inconshreveable/go-vhost v0.0.0-20160627193104-06d84117953b</span><br><span class="line">go: downloading github.com/google/go-github v17.0.0+incompatible</span><br><span class="line">go: downloading github.com/gorilla/websocket v1.4.1</span><br><span class="line">go: extracting github.com/antchfx/xpath v1.1.0</span><br><span class="line">go: extracting github.com/bettercap/recording v0.0.0-20190408083647-3ce1dcf032e3</span><br><span class="line">go: downloading github.com/gorilla/mux v1.7.3</span><br><span class="line">go: extracting github.com/gorilla/websocket v1.4.1</span><br><span class="line">go: downloading github.com/bettercap/gatt v0.0.0-20191018133023-569d3d9372bb</span><br><span class="line">go: extracting github.com/google/gopacket v1.1.17</span><br><span class="line">go: downloading golang.org/x/net v0.0.0-20191014212845-da9a3fd4c582</span><br><span class="line">go: extracting github.com/robertkrimen/otto v0.0.0-20180617131154-15f95af6e78d</span><br><span class="line">go: extracting github.com/gorilla/mux v1.7.3</span><br><span class="line">go: downloading github.com/adrianmo/go-nmea v1.1.0</span><br><span class="line">go: downloading github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07</span><br><span class="line">go: extracting github.com/google/go-github v17.0.0+incompatible</span><br><span class="line">go: extracting github.com/bettercap/gatt v0.0.0-20191018133023-569d3d9372bb</span><br><span class="line">go: downloading golang.org/x/sys v0.0.0-20191018095205-727590c5006e</span><br><span class="line">go: downloading github.com/kr/binarydist v0.1.0</span><br><span class="line">go: downloading github.com/mdlayher/dhcp6 v0.0.0-20190311162359-2a67805d7d0b</span><br><span class="line">go: extracting github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07</span><br><span class="line">go: extracting github.com/adrianmo/go-nmea v1.1.0</span><br><span class="line">go: extracting github.com/kr/binarydist v0.1.0</span><br><span class="line">go: downloading github.com/hashicorp/mdns v1.0.1</span><br><span class="line">go: downloading github.com/elazarl/goproxy v0.0.0-20191011121108-aa519ddbe484</span><br><span class="line">go: downloading golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550</span><br><span class="line">go: downloading gopkg.in/sourcemap.v1 v1.0.5</span><br><span class="line">go: extracting github.com/mdlayher/dhcp6 v0.0.0-20190311162359-2a67805d7d0b</span><br><span class="line">go: downloading github.com/google/go-querystring v1.0.0</span><br><span class="line">go: downloading github.com/mgutz/logxi v0.0.0-20161027140823-aebf8a7d67ab</span><br><span class="line">go: extracting github.com/hashicorp/mdns v1.0.1</span><br><span class="line">go: extracting gopkg.in/sourcemap.v1 v1.0.5</span><br><span class="line">go: extracting github.com/elazarl/goproxy v0.0.0-20191011121108-aa519ddbe484</span><br><span class="line">go: downloading github.com/mitchellh/go-homedir v1.1.0</span><br><span class="line">go: extracting github.com/google/go-querystring v1.0.0</span><br><span class="line">go: extracting golang.org/x/net v0.0.0-20191014212845-da9a3fd4c582</span><br><span class="line">go: extracting github.com/mitchellh/go-homedir v1.1.0</span><br><span class="line">go: extracting github.com/mgutz/logxi v0.0.0-20161027140823-aebf8a7d67ab</span><br><span class="line">go: downloading github.com/mattn/go-isatty v0.0.10</span><br><span class="line">go: downloading github.com/mattn/go-colorable v0.1.4</span><br><span class="line">go: downloading github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b</span><br><span class="line">go: extracting github.com/mattn/go-isatty v0.0.10</span><br><span class="line">go: extracting github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b</span><br><span class="line">go: extracting github.com/mattn/go-colorable v0.1.4</span><br><span class="line">go: extracting golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550</span><br><span class="line">go: extracting golang.org/x/sys v0.0.0-20191018095205-727590c5006e</span><br><span class="line">go: finding github.com/evilsocket/islazy v1.10.4</span><br><span class="line">go: finding github.com/malfunkt/iprange v0.9.0</span><br><span class="line">go: finding github.com/dustin/go-humanize v1.0.0</span><br><span class="line">go: finding github.com/mitchellh/go-homedir v1.1.0</span><br><span class="line">go: finding github.com/adrianmo/go-nmea v1.1.0</span><br><span class="line">go: finding github.com/bettercap/readline v0.0.0-20180208083827-9cec905dd291</span><br><span class="line">go: finding github.com/bettercap/gatt v0.0.0-20191018133023-569d3d9372bb</span><br><span class="line">go: finding github.com/bettercap/recording v0.0.0-20190408083647-3ce1dcf032e3</span><br><span class="line">go: finding github.com/gorilla/mux v1.7.3</span><br><span class="line">go: finding github.com/mgutz/logxi v0.0.0-20161027140823-aebf8a7d67ab</span><br><span class="line">go: finding github.com/google/gopacket v1.1.17</span><br><span class="line">go: finding github.com/tarm/serial v0.0.0-20180830185346-98f6abe2eb07</span><br><span class="line">go: finding github.com/gorilla/websocket v1.4.1</span><br><span class="line">go: finding github.com/antchfx/jsonquery v1.0.0</span><br><span class="line">go: finding github.com/mattn/go-colorable v0.1.4</span><br><span class="line">go: finding github.com/gobwas/glob v0.0.0-20181002190808-e7a84e9525fe</span><br><span class="line">go: finding github.com/bettercap/nrf24 v0.0.0-20190219153547-aa37e6d0e0eb</span><br><span class="line">go: finding github.com/mdlayher/dhcp6 v0.0.0-20190311162359-2a67805d7d0b</span><br><span class="line">go: finding github.com/pkg/errors v0.8.1</span><br><span class="line">go: finding github.com/mattn/go-isatty v0.0.10</span><br><span class="line">go: finding github.com/robertkrimen/otto v0.0.0-20180617131154-15f95af6e78d</span><br><span class="line">go: finding golang.org/x/sys v0.0.0-20191018095205-727590c5006e</span><br><span class="line">go: finding github.com/google/gousb v0.0.0-20190812193832-18f4c1d8a750</span><br><span class="line">go: finding github.com/kr/binarydist v0.1.0</span><br><span class="line">go: finding github.com/antchfx/xpath v1.1.0</span><br><span class="line">go: finding github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b</span><br><span class="line">go: finding github.com/hashicorp/mdns v1.0.1</span><br><span class="line">go: finding github.com/elazarl/goproxy v0.0.0-20191011121108-aa519ddbe484</span><br><span class="line">go: finding github.com/miekg/dns v1.1.22</span><br><span class="line">go: finding github.com/google/go-github v17.0.0+incompatible</span><br><span class="line">go: finding golang.org/x/net v0.0.0-20191014212845-da9a3fd4c582</span><br><span class="line">go: finding gopkg.in/sourcemap.v1 v1.0.5</span><br><span class="line">go: finding github.com/inconshreveable/go-vhost v0.0.0-20160627193104-06d84117953b</span><br><span class="line">go: finding github.com/jpillora/go-tld v0.0.0-20190202073305-f16ca3b7b383</span><br><span class="line">go: finding github.com/google/go-querystring v1.0.0</span><br><span class="line">go: finding golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550</span><br><span class="line">Removing intermediate container 1faa1d85eb96</span><br><span class="line"> ---> b3d614b2ec69</span><br><span class="line">Step 8/17 : RUN mkdir -p /usr/local/share/bettercap</span><br><span class="line"> ---> Running in 14eb7daf4ff8</span><br><span class="line">Removing intermediate container 14eb7daf4ff8</span><br><span class="line"> ---> 68727ce1be68</span><br><span class="line">Step 9/17 : RUN git clone https://github.com/bettercap/caplets /usr/local/share/bettercap/caplets</span><br><span class="line"> ---> Running in be1c4c4a54cc</span><br><span class="line">Cloning into '/usr/local/share/bettercap/caplets'...</span><br><span class="line">Removing intermediate container be1c4c4a54cc</span><br><span class="line"> ---> 2c4f0fa1e524</span><br><span class="line">Step 10/17 : FROM alpine</span><br><span class="line">latest: Pulling from library/alpine</span><br><span class="line">99fc70ac0b64: Already exists </span><br><span class="line">Digest: sha256:c19173c5ada610a5989151111163d28a67368362762534d8a8121ce95cf2bd5a</span><br><span class="line">Status: Downloaded newer image for alpine:latest</span><br><span class="line"> ---> 27560c0c9c97</span><br><span class="line">Step 11/17 : RUN apk add --update ca-certificates</span><br><span class="line"> ---> Running in 509fe0867c6d</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/armv7/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/armv7/APKINDEX.tar.gz</span><br><span class="line">(1/1) Installing ca-certificates (20190108-r0)</span><br><span class="line">Executing busybox-1.30.1-r2.trigger</span><br><span class="line">Executing ca-certificates-20190108-r0.trigger</span><br><span class="line">OK: 4 MiB in 15 packages</span><br><span class="line">Removing intermediate container 509fe0867c6d</span><br><span class="line"> ---> 9293f9271587</span><br><span class="line">Step 12/17 : RUN apk add --no-cache --update bash iproute2 libpcap libusb-dev libnetfilter_queue wireless-tools</span><br><span class="line"> ---> Running in 65daa4f9d22b</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/main/armv7/APKINDEX.tar.gz</span><br><span class="line">fetch http://dl-cdn.alpinelinux.org/alpine/v3.10/community/armv7/APKINDEX.tar.gz</span><br><span class="line">(1/16) Installing ncurses-terminfo-base (6.1_p20190518-r0)</span><br><span class="line">(2/16) Installing ncurses-terminfo (6.1_p20190518-r0)</span><br><span class="line">(3/16) Installing ncurses-libs (6.1_p20190518-r0)</span><br><span class="line">(4/16) Installing readline (8.0.0-r0)</span><br><span class="line">(5/16) Installing bash (5.0.0-r0)</span><br><span class="line">Executing bash-5.0.0-r0.post-install</span><br><span class="line">(6/16) Installing libmnl (1.0.4-r0)</span><br><span class="line">(7/16) Installing libnftnl-libs (1.1.3-r0)</span><br><span class="line">(8/16) Installing iptables (1.8.3-r0)</span><br><span class="line">(9/16) Installing iproute2 (4.20.0-r1)</span><br><span class="line">Executing iproute2-4.20.0-r1.post-install</span><br><span class="line">(10/16) Installing libnfnetlink (1.0.1-r1)</span><br><span class="line">(11/16) Installing libnetfilter_queue (1.0.3-r0)</span><br><span class="line">(12/16) Installing libpcap (1.9.1-r0)</span><br><span class="line">(13/16) Installing libusb (1.0.22-r0)</span><br><span class="line">(14/16) Installing pkgconf (1.6.1-r1)</span><br><span class="line">(15/16) Installing libusb-dev (1.0.22-r0)</span><br><span class="line">(16/16) Installing wireless-tools (30_pre9-r1)</span><br><span class="line">Executing busybox-1.30.1-r2.trigger</span><br><span class="line">OK: 16 MiB in 31 packages</span><br><span class="line">Removing intermediate container 65daa4f9d22b</span><br><span class="line"> ---> de5bed4329f4</span><br><span class="line">Step 13/17 : COPY --from=build-env /go/src/github.com/bettercap/bettercap/bettercap /app/</span><br><span class="line"> ---> 7509d206b9bb</span><br><span class="line">Step 14/17 : COPY --from=build-env /usr/local/share/bettercap/caplets /app/</span><br><span class="line"> ---> 867af90533b0</span><br><span class="line">Step 15/17 : WORKDIR /app</span><br><span class="line"> ---> Running in e6badf626501</span><br><span class="line">Removing intermediate container e6badf626501</span><br><span class="line"> ---> c8d068058c77</span><br><span class="line">Step 16/17 : EXPOSE 80 443 53 5300 8080 8081 8082 8083 8000</span><br><span class="line"> ---> Running in dee9fe5c9c12</span><br><span class="line">Removing intermediate container dee9fe5c9c12</span><br><span class="line"> ---> e09ab03d9632</span><br><span class="line">Step 17/17 : ENTRYPOINT ["/app/bettercap"]</span><br><span class="line"> ---> Running in 056b18a9cf33</span><br><span class="line">Removing intermediate container 056b18a9cf33</span><br><span class="line"> ---> fa1faf86b2e9</span><br><span class="line">Successfully built fa1faf86b2e9</span><br></pre></td></tr></table></figure><p>it shows I successfully build an image.</p><a id="more"></a><h2 id="push-the-image-to-dockerhub-repo"><a href="#push-the-image-to-dockerhub-repo" class="headerlink" title="push the image to dockerhub repo"></a>push the image to dockerhub repo</h2><p>I need to create a dockerhub account first then log in.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker login</span><br><span class="line">Login with your Docker ID to push and pull images from Docker Hub. If you don't have a Docker ID, head over to https://hub.docker.com to create one.</span><br><span class="line">Username: vmnet8</span><br><span class="line">Password: </span><br><span class="line">WARNING! Your password will be stored unencrypted in /root/.docker/config.json.</span><br><span class="line">Configure a credential helper to remove this warning. See</span><br><span class="line">https://docs.docker.com/engine/reference/commandline/login/#credentials-store</span><br><span class="line"></span><br><span class="line">Login Succeeded</span><br></pre></td></tr></table></figure><p>On dockhub.com, create new repo, and name the repo</p><img src="/2019/12/06/Build-a-docker-image-bettercap/create_repo.png" class=""><p>create a tag for my image</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker image tag fa1faf86b2e9 vmnet8/bettercap:first</span><br></pre></td></tr></table></figure><p>check if it works</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">vmnet8/bettercap first fa1faf86b2e9 45 hours ago 35.9MB</span><br></pre></td></tr></table></figure><p>push this image to dockerhub</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker push vmnet8/bettercap:first</span><br><span class="line">The push refers to repository [docker.io/vmnet8/bettercap]</span><br><span class="line">6932e1802232: Pushed </span><br><span class="line">1c42bc244f9a: Pushed </span><br><span class="line">316457fe74cb: Pushed </span><br><span class="line">dbba7885a7cb: Pushed </span><br><span class="line">30a0150d2171: Pushed </span><br><span class="line">first: digest: sha256:42cb2a552f6b3f4b11f2d9514fd8d8e4288af7060bd148ec78fc4931551b89dc size: 1372</span><br></pre></td></tr></table></figure><img src="/2019/12/06/Build-a-docker-image-bettercap/in_repo.png" class=""><h2 id="docker-manifest"><a href="#docker-manifest" class="headerlink" title="docker manifest"></a>docker manifest</h2><p>The docker manifest command gives users additional information such as the os and architecture an image was built for.<br>A manifest list is a list of image layers that is created by specifying one or more (ideally more than one) image names.<br>Ideally a manifest list is created from images that are identical in function for different os/arch combinations. For this reason, manifest lists are often referred to as “multi-arch images”. However, a user could create a manifest list that points to two images – one for windows on amd64, and one for darwin on amd64.</p><p>manifest inspect</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker manifest inspect -v vmnet8/bettercap:first</span><br><span class="line">{</span><br><span class="line">"Ref": "docker.io/vmnet8/bettercap:first",</span><br><span class="line">"Descriptor": {</span><br><span class="line">"mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line">"digest": "sha256:42cb2a552f6b3f4b11f2d9514fd8d8e4288af7060bd148ec78fc4931551b89dc",</span><br><span class="line">"size": 1372,</span><br><span class="line">"platform": {</span><br><span class="line">"architecture": "arm",</span><br><span class="line">"os": "linux"</span><br><span class="line">}</span><br><span class="line">},</span><br><span class="line">"SchemaV2Manifest": {</span><br><span class="line">"schemaVersion": 2,</span><br><span class="line">"mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line">"config": {</span><br><span class="line">"mediaType": "application/vnd.docker.container.image.v1+json",</span><br><span class="line">"size": 3064,</span><br><span class="line">"digest": "sha256:fa1faf86b2e927540757fbf1da0cabc747ef90de3ba0ac79b92c3a2524aa30e0"</span><br><span class="line">},</span><br><span class="line">"layers": [</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 2378437,</span><br><span class="line">"digest": "sha256:99fc70ac0b64db67086f98ceb3942600816eed98046abd6be5ad66f4614a9ca2"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 1674759,</span><br><span class="line">"digest": "sha256:9870f62c5af97178d2b7f26ef09285c407bb8f86e1be9bcf9d82aeb5a036b7f6"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 2445148,</span><br><span class="line">"digest": "sha256:cbb20a312883952b6b42b5f71171308cc5e7a1183bad442878febba39a7ae4c1"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 11138965,</span><br><span class="line">"digest": "sha256:b998c1e6578adab28ed6958da133eeff1b72412d0be30ca5b2dc7603eb906ddb"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 616532,</span><br><span class="line">"digest": "sha256:17c2d33598da93c28bc860cff35445cb8335a12d0c2c102ec15bd2fccba5e24b"</span><br><span class="line">}</span><br><span class="line">]</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>create a manifest</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker manifest create vmnet8/bettercap:latest vmnet8/bettercap:linux-arm</span><br><span class="line">Created manifest list docker.io/vmnet8/bettercap:latest</span><br></pre></td></tr></table></figure><p>manifest inspect</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker manifest inspect -v vmnet8/bettercap:linux-arm</span><br><span class="line">{</span><br><span class="line">"Ref": "docker.io/vmnet8/bettercap:linux-arm",</span><br><span class="line">"Descriptor": {</span><br><span class="line">"mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line">"digest": "sha256:42cb2a552f6b3f4b11f2d9514fd8d8e4288af7060bd148ec78fc4931551b89dc",</span><br><span class="line">"size": 1372,</span><br><span class="line">"platform": {</span><br><span class="line">"architecture": "arm",</span><br><span class="line">"os": "linux"</span><br><span class="line">}</span><br><span class="line">},</span><br><span class="line">"SchemaV2Manifest": {</span><br><span class="line">"schemaVersion": 2,</span><br><span class="line">"mediaType": "application/vnd.docker.distribution.manifest.v2+json",</span><br><span class="line">"config": {</span><br><span class="line">"mediaType": "application/vnd.docker.container.image.v1+json",</span><br><span class="line">"size": 3064,</span><br><span class="line">"digest": "sha256:fa1faf86b2e927540757fbf1da0cabc747ef90de3ba0ac79b92c3a2524aa30e0"</span><br><span class="line">},</span><br><span class="line">"layers": [</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 2378437,</span><br><span class="line">"digest": "sha256:99fc70ac0b64db67086f98ceb3942600816eed98046abd6be5ad66f4614a9ca2"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 1674759,</span><br><span class="line">"digest": "sha256:9870f62c5af97178d2b7f26ef09285c407bb8f86e1be9bcf9d82aeb5a036b7f6"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 2445148,</span><br><span class="line">"digest": "sha256:cbb20a312883952b6b42b5f71171308cc5e7a1183bad442878febba39a7ae4c1"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 11138965,</span><br><span class="line">"digest": "sha256:b998c1e6578adab28ed6958da133eeff1b72412d0be30ca5b2dc7603eb906ddb"</span><br><span class="line">},</span><br><span class="line">{</span><br><span class="line">"mediaType": "application/vnd.docker.image.rootfs.diff.tar.gzip",</span><br><span class="line">"size": 616532,</span><br><span class="line">"digest": "sha256:17c2d33598da93c28bc860cff35445cb8335a12d0c2c102ec15bd2fccba5e24b"</span><br><span class="line">}</span><br><span class="line">]</span><br><span class="line">}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>manifest push</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">root@vmnet8cloud:~/bettercap/bettercap# docker manifest push vmnet8/bettercap:latest</span><br><span class="line">sha256:2b6c7c74af114f187139ff8bdacc61af79a9a1715f03835efb887a77ac16f17d</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> bettercap </tag>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Bettercap project: install on raspberry Pi</title>
<link href="/2019/11/23/Bettercap-project-install-on-raspberry-Pi/"/>
<url>/2019/11/23/Bettercap-project-install-on-raspberry-Pi/</url>
<content type="html"><![CDATA[<h2 id="install-on-raspberry-Pi3B"><a href="#install-on-raspberry-Pi3B" class="headerlink" title="install on raspberry Pi3B+"></a>install on raspberry Pi3B+</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# sudo apt update</span><br><span class="line">Hit:1 https://download.docker.com/linux/raspbian stretch InRelease</span><br><span class="line">Get:2 http://archive.raspberrypi.org/debian buster InRelease [25.2 kB] </span><br><span class="line">Hit:3 https://deb.nodesource.com/node_10.x buster InRelease </span><br><span class="line">Get:4 http://deb.torproject.org/torproject.org buster InRelease [4,964 B] </span><br><span class="line">Get:5 http://raspbian.raspberrypi.org/raspbian buster InRelease [15.0 kB] </span><br><span class="line">Get:6 https://packages.cloud.google.com/apt coral-cloud-stable InRelease [6,328 B] </span><br><span class="line">Get:7 http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages [13.0 MB]</span><br><span class="line">Get:8 http://archive.raspberrypi.org/debian buster/main armhf Packages [259 kB]</span><br><span class="line">Ign:9 https://packages.cloud.google.com/apt coral-cloud-stable/main armhf Packages</span><br><span class="line">Get:9 https://packages.cloud.google.com/apt coral-cloud-stable/main armhf Packages [3,358 B]</span><br><span class="line">Fetched 13.3 MB in 25s (539 kB/s) </span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">53 packages can be upgraded. Run 'apt list --upgradable' to see them.</span><br></pre></td></tr></table></figure><p>next</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# sudo apt install golang git build-essential libpcap-dev libusb-1.0-0-dev libnetfilter-queue-dev</span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">build-essential is already the newest version (12.6).</span><br><span class="line">git is already the newest version (1:2.20.1-2).</span><br><span class="line">The following package was automatically installed and is no longer required:</span><br><span class="line"> point-rpi</span><br><span class="line">Use 'sudo apt autoremove' to remove it.</span><br><span class="line">The following additional packages will be installed:</span><br><span class="line"> golang-1.11 golang-1.11-doc golang-1.11-go golang-1.11-src golang-doc golang-go golang-src libnfnetlink-dev libpcap0.8-dev libusb-1.0-doc</span><br><span class="line">Suggested packages:</span><br><span class="line"> bzr | brz mercurial subversion</span><br><span class="line">The following NEW packages will be installed:</span><br><span class="line"> golang golang-1.11 golang-1.11-doc golang-1.11-go golang-1.11-src golang-doc golang-go golang-src libnetfilter-queue-dev libnfnetlink-dev</span><br><span class="line"> libpcap-dev libpcap0.8-dev libusb-1.0-0-dev libusb-1.0-doc</span><br><span class="line">0 upgraded, 14 newly installed, 0 to remove and 53 not upgraded.</span><br><span class="line">Need to get 52.4 MB of archives.</span><br><span class="line">After this operation, 260 MB of additional disk space will be used.</span><br><span class="line">Do you want to continue? [Y/n] y</span><br><span class="line">Get:1 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-1.11-src armhf 1.11.6-1+rpi1+deb10u2 [13.0 MB]</span><br><span class="line">Get:2 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-1.11-go armhf 1.11.6-1+rpi1+deb10u2 [36.3 MB] </span><br><span class="line">Get:3 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-1.11-doc all 1.11.6-1+rpi1+deb10u2 [2,517 kB] </span><br><span class="line">Get:4 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-1.11 all 1.11.6-1+rpi1+deb10u2 [24.1 kB] </span><br><span class="line">Get:5 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-src armhf 2:1.11~1+b6 [4,660 B] </span><br><span class="line">Get:6 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-go armhf 2:1.11~1+b6 [23.7 kB] </span><br><span class="line">Get:7 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang-doc all 2:1.11~1 [4,392 B] </span><br><span class="line">Get:8 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf golang armhf 2:1.11~1+b6 [4,612 B] </span><br><span class="line">Get:9 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libnfnetlink-dev armhf 1.0.1-3 [7,928 B] </span><br><span class="line">Get:10 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libnetfilter-queue-dev armhf 1.0.3-1 [6,548 B] </span><br><span class="line">Get:11 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libpcap0.8-dev armhf 1.8.1-6 [220 kB] </span><br><span class="line">Get:12 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libpcap-dev armhf 1.8.1-6 [25.9 kB] </span><br><span class="line">Get:13 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libusb-1.0-0-dev armhf 2:1.0.22-2 [65.5 kB] </span><br><span class="line">Get:14 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libusb-1.0-doc all 2:1.0.22-2 [182 kB] </span><br><span class="line">Fetched 52.4 MB in 23s (2,250 kB/s) </span><br><span class="line">Selecting previously unselected package golang-1.11-src.</span><br><span class="line">(Reading database ... 154643 files and directories currently installed.)</span><br><span class="line">Preparing to unpack .../00-golang-1.11-src_1.11.6-1+rpi1+deb10u2_armhf.deb ...</span><br><span class="line">Unpacking golang-1.11-src (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Selecting previously unselected package golang-1.11-go.</span><br><span class="line">Preparing to unpack .../01-golang-1.11-go_1.11.6-1+rpi1+deb10u2_armhf.deb ...</span><br><span class="line">Unpacking golang-1.11-go (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Selecting previously unselected package golang-1.11-doc.</span><br><span class="line">Preparing to unpack .../02-golang-1.11-doc_1.11.6-1+rpi1+deb10u2_all.deb ...</span><br><span class="line">Unpacking golang-1.11-doc (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Selecting previously unselected package golang-1.11.</span><br><span class="line">Preparing to unpack .../03-golang-1.11_1.11.6-1+rpi1+deb10u2_all.deb ...</span><br><span class="line">Unpacking golang-1.11 (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Selecting previously unselected package golang-src.</span><br><span class="line">Preparing to unpack .../04-golang-src_2%3a1.11~1+b6_armhf.deb ...</span><br><span class="line">Unpacking golang-src (2:1.11~1+b6) ...</span><br><span class="line">Selecting previously unselected package golang-go.</span><br><span class="line">Preparing to unpack .../05-golang-go_2%3a1.11~1+b6_armhf.deb ...</span><br><span class="line">Unpacking golang-go (2:1.11~1+b6) ...</span><br><span class="line">Selecting previously unselected package golang-doc.</span><br><span class="line">Preparing to unpack .../06-golang-doc_2%3a1.11~1_all.deb ...</span><br><span class="line">Unpacking golang-doc (2:1.11~1) ...</span><br><span class="line">Selecting previously unselected package golang.</span><br><span class="line">Preparing to unpack .../07-golang_2%3a1.11~1+b6_armhf.deb ...</span><br><span class="line">Unpacking golang (2:1.11~1+b6) ...</span><br><span class="line">Selecting previously unselected package libnfnetlink-dev.</span><br><span class="line">Preparing to unpack .../08-libnfnetlink-dev_1.0.1-3_armhf.deb ...</span><br><span class="line">Unpacking libnfnetlink-dev (1.0.1-3) ...</span><br><span class="line">Selecting previously unselected package libnetfilter-queue-dev.</span><br><span class="line">Preparing to unpack .../09-libnetfilter-queue-dev_1.0.3-1_armhf.deb ...</span><br><span class="line">Unpacking libnetfilter-queue-dev (1.0.3-1) ...</span><br><span class="line">Selecting previously unselected package libpcap0.8-dev:armhf.</span><br><span class="line">Preparing to unpack .../10-libpcap0.8-dev_1.8.1-6_armhf.deb ...</span><br><span class="line">Unpacking libpcap0.8-dev:armhf (1.8.1-6) ...</span><br><span class="line">Selecting previously unselected package libpcap-dev:armhf.</span><br><span class="line">Preparing to unpack .../11-libpcap-dev_1.8.1-6_armhf.deb ...</span><br><span class="line">Unpacking libpcap-dev:armhf (1.8.1-6) ...</span><br><span class="line">Selecting previously unselected package libusb-1.0-0-dev:armhf.</span><br><span class="line">Preparing to unpack .../12-libusb-1.0-0-dev_2%3a1.0.22-2_armhf.deb ...</span><br><span class="line">Unpacking libusb-1.0-0-dev:armhf (2:1.0.22-2) ...</span><br><span class="line">Selecting previously unselected package libusb-1.0-doc.</span><br><span class="line">Preparing to unpack .../13-libusb-1.0-doc_2%3a1.0.22-2_all.deb ...</span><br><span class="line">Unpacking libusb-1.0-doc (2:1.0.22-2) ...</span><br><span class="line">Setting up libnfnetlink-dev (1.0.1-3) ...</span><br><span class="line">Setting up golang-1.11-src (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Setting up libpcap0.8-dev:armhf (1.8.1-6) ...</span><br><span class="line">Setting up libusb-1.0-doc (2:1.0.22-2) ...</span><br><span class="line">Setting up libusb-1.0-0-dev:armhf (2:1.0.22-2) ...</span><br><span class="line">Setting up golang-1.11-go (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Setting up libnetfilter-queue-dev (1.0.3-1) ...</span><br><span class="line">Setting up golang-src (2:1.11~1+b6) ...</span><br><span class="line">Setting up libpcap-dev:armhf (1.8.1-6) ...</span><br><span class="line">Setting up golang-1.11-doc (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Setting up golang-go (2:1.11~1+b6) ...</span><br><span class="line">Setting up golang-1.11 (1.11.6-1+rpi1+deb10u2) ...</span><br><span class="line">Setting up golang-doc (2:1.11~1) ...</span><br><span class="line">Setting up golang (2:1.11~1+b6) ...</span><br><span class="line">Processing triggers for man-db (2.8.5-2) ...</span><br></pre></td></tr></table></figure><p>next</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# go get -u github.com/bettercap/bettercap</span><br></pre></td></tr></table></figure><p>Once the build process is concluded, the binary will be located in go/bin/bettercap<br>I checked the go/bin/bettercap</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# which go/bin/bettercap</span><br><span class="line">go/bin/bettercap</span><br></pre></td></tr></table></figure><a id="more"></a><h2 id="start-bettercap"><a href="#start-bettercap" class="headerlink" title="start bettercap"></a>start bettercap</h2><p>need to make sure bettercap command under /usr/bin</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# cp go/bin/bettercap /usr/bin</span><br><span class="line">root@treehouses:~# which bettercap</span><br><span class="line">/usr/bin/bettercap</span><br></pre></td></tr></table></figure><p>run following command</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# bettercap -eval "caplets.update; ui.update; q"</span><br><span class="line">bettercap v2.26.1 (built for linux arm with go1.11.6) [type 'help' for a list of commands]</span><br><span class="line"></span><br><span class="line">[00:07:08] [sys.log] [inf] caplets creating caplets install path /usr/local/share/bettercap/ ...</span><br><span class="line">[00:07:08] [sys.log] [inf] caplets downloading caplets from https://github.com/bettercap/caplets/archive/master.zip ...</span><br><span class="line">[00:07:10] [sys.log] [inf] caplets installing caplets to /usr/local/share/bettercap/caplets ...</span><br><span class="line">[00:07:10] [sys.log] [inf] ui checking latest stable release ...</span><br><span class="line">[00:07:10] [sys.log] [inf] ui downloading ui v1.3.0 from https://github.com/bettercap/ui/releases/download/v1.3.0/ui.zip ...</span><br><span class="line">[00:07:14] [sys.log] [inf] ui installing to /usr/local/share/bettercap/ui ...</span><br><span class="line">[00:07:18] [sys.log] [inf] ui installation complete, you can now run the http-ui (or https-ui) caplet to start the UI.</span><br></pre></td></tr></table></figure><p>Inorder to login to web UI, I need do check this first</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~ $ cat /usr/local/share/bettercap/caplets/https-ui.cap</span><br><span class="line"># api listening on https://0.0.0.0:8083/ and ui on https://0.0.0.0</span><br><span class="line">set api.rest.address 0.0.0.0</span><br><span class="line">set api.rest.port 8083</span><br><span class="line">set https.server.address 0.0.0.0</span><br><span class="line">set https.server.port 443</span><br><span class="line"></span><br><span class="line">## make sure both use the same https certificate so api requests won't fail</span><br><span class="line">set https.server.certificate ~/.bettercap-https.cert.pem</span><br><span class="line">set https.server.key ~/.bettercap-https.key.pem </span><br><span class="line">set api.rest.certificate ~/.bettercap-https.cert.pem</span><br><span class="line">set api.rest.key ~/.bettercap-https.key.pem </span><br><span class="line"># default installation path of the ui</span><br><span class="line">set https.server.path /usr/local/share/bettercap/ui</span><br><span class="line"></span><br><span class="line"># !!! CHANGE THESE !!!</span><br><span class="line">set api.rest.username user</span><br><span class="line">set api.rest.password pass</span><br><span class="line"></span><br><span class="line"># go!</span><br><span class="line">api.rest on</span><br><span class="line">https.server on</span><br></pre></td></tr></table></figure><h2 id="start-bettercap-1"><a href="#start-bettercap-1" class="headerlink" title="start bettercap"></a>start bettercap</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# bettercap -caplet https-ui</span><br><span class="line">bettercap v2.26.1 (built for linux arm with go1.11.6) [type 'help' for a list of commands]</span><br><span class="line"></span><br><span class="line">[00:17:17] [sys.log] [inf] api.rest generating TLS key to /root/.bettercap-https.key.pem</span><br><span class="line">[00:17:17] [sys.log] [inf] api.rest generating TLS certificate to /root/.bettercap-https.cert.pem</span><br><span class="line">192.168.0.0/24 > 192.168.0.33 » [00:18:44] [sys.log] [inf] https.server loading server TLS certificate from /root/.bettercap-https.cert.pem</span><br><span class="line">192.168.0.0/24 > 192.168.0.33 » [00:18:44] [sys.log] [inf] https.server loading server TLS key from /root/.bettercap-https.key.pem</span><br><span class="line">192.168.0.0/24 > 192.168.0.33 » [00:18:44] [sys.log] [inf] https.server starting on https://0.0.0.0:443</span><br><span class="line">192.168.0.0/24 > 192.168.0.33 » [00:18:44] [sys.log] [inf] api.rest api server starting on https://0.0.0.0:8083</span><br><span class="line">192.168.0.0/24 > 192.168.0.33 » 2019/11/23 00:20:07 http: TLS handshake error from 192.168.0.126:46472: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:07 http: TLS handshake error from 192.168.0.126:46474: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:08 http: TLS handshake error from 192.168.0.126:46486: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:08 http: TLS handshake error from 192.168.0.126:46488: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:14 http: TLS handshake error from 192.168.0.126:46528: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:14 http: TLS handshake error from 192.168.0.126:46530: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:21 http: TLS handshake error from 192.168.0.126:46574: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:21 http: TLS handshake error from 192.168.0.126:46576: remote error: tls: unknown certificate</span><br><span class="line">2019/11/23 00:20:53 http: TLS handshake error from 192.168.0.126:60944: remote error: tls: unknown certificate</span><br></pre></td></tr></table></figure><p>then login <a href="192.168.0.33">https://192.168.0.33</a>, using username and password</p><img src="/2019/11/23/Bettercap-project-install-on-raspberry-Pi/bettercap_pi_install.png" class="">]]></content>
<tags>
<tag> bettercap </tag>
</tags>
</entry>
<entry>
<title>Bettercap project: install on local Ubuntu</title>
<link href="/2019/11/22/Bettercap-project_install_Ubuntu/"/>
<url>/2019/11/22/Bettercap-project_install_Ubuntu/</url>
<content type="html"><![CDATA[<h2 id="install-on-Ubuntu"><a href="#install-on-Ubuntu" class="headerlink" title="install on Ubuntu"></a>install on Ubuntu</h2><p>copy the shell script to install bettercap, then chmod +x, then run it</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~$ ./bettercap.sh </span><br><span class="line">--2019-11-20 19:48:44-- https://github.com/bettercap/bettercap/releases/download/v2.26/bettercap_linux_amd64_v2.26.zip</span><br><span class="line">Resolving github.com (github.com)... 192.30.255.113</span><br><span class="line">Connecting to github.com (github.com)|192.30.255.113|:443... connected.</span><br><span class="line">HTTP request sent, awaiting response... 302 Found</span><br><span class="line">Location: https://github-production-release-asset-2e65be.s3.amazonaws.com/116576783/09ae2a00-f1be-11e9-9887-8cb5e818dc9b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191121%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191121T034844Z&X-Amz-Expires=300&X-Amz-Signature=622884cd807d933374563a875deaf3f0c5dd00788d6324e5b4fc5e07cc724198&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dbettercap_linux_amd64_v2.26.zip&response-content-type=application%2Foctet-stream [following]</span><br><span class="line">--2019-11-20 19:48:44-- https://github-production-release-asset-2e65be.s3.amazonaws.com/116576783/09ae2a00-f1be-11e9-9887-8cb5e818dc9b?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIWNJYAX4CSVEH53A%2F20191121%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20191121T034844Z&X-Amz-Expires=300&X-Amz-Signature=622884cd807d933374563a875deaf3f0c5dd00788d6324e5b4fc5e07cc724198&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Dbettercap_linux_amd64_v2.26.zip&response-content-type=application%2Foctet-stream</span><br><span class="line">Resolving github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)... 52.216.25.140</span><br><span class="line">Connecting to github-production-release-asset-2e65be.s3.amazonaws.com (github-production-release-asset-2e65be.s3.amazonaws.com)|52.216.25.140|:443... connected.</span><br><span class="line">HTTP request sent, awaiting response... 200 OK</span><br><span class="line">Length: 14037021 (13M) [application/octet-stream]</span><br><span class="line">Saving to: ‘bettercap_linux_amd64_v2.26.zip’</span><br><span class="line"></span><br><span class="line">bettercap_linux_amd64_v2.26.zip 100%[=================================================================>] 13.39M 1.39MB/s in 9.9s </span><br><span class="line"></span><br><span class="line">2019-11-20 19:48:55 (1.35 MB/s) - ‘bettercap_linux_amd64_v2.26.zip’ saved [14037021/14037021]</span><br><span class="line"></span><br><span class="line">Archive: bettercap_linux_amd64_v2.26.zip</span><br><span class="line"> inflating: bettercap </span><br><span class="line"> inflating: bettercap_linux_amd64_v2.26.sha256</span><br><span class="line">[sudo] password for anna: </span><br><span class="line"></span><br><span class="line"></span><br><span class="line">bettercap: error while loading shared libraries: libnetfilter_queue.so.1: cannot open shared object file: No such file or directory</span><br></pre></td></tr></table></figure><p>It shows something wrong during installation, so I try the last command</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~$ sudo bettercap -eval "caplets.update; ui.update; quit"</span><br><span class="line">bettercap: error while loading shared libraries: libnetfilter_queue.so.1: cannot open shared object file: No such file or directory</span><br></pre></td></tr></table></figure><p>It shows I need to install libnetfilter-queue1</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~$ sudo apt install libnetfilter-queue1</span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">The following NEW packages will be installed:</span><br><span class="line"> libnetfilter-queue1</span><br><span class="line">0 upgraded, 1 newly installed, 0 to remove and 35 not upgraded.</span><br><span class="line">Need to get 11.4 kB of archives.</span><br><span class="line">After this operation, 61.4 kB of additional disk space will be used.</span><br><span class="line">Get:1 http://us.archive.ubuntu.com/ubuntu bionic/universe amd64 libnetfilter-queue1 amd64 1.0.2-2 [11.4 kB]</span><br><span class="line">Fetched 11.4 kB in 0s (42.2 kB/s) </span><br><span class="line">Selecting previously unselected package libnetfilter-queue1.</span><br><span class="line">(Reading database ... 219919 files and directories currently installed.)</span><br><span class="line">Preparing to unpack .../libnetfilter-queue1_1.0.2-2_amd64.deb ...</span><br><span class="line">Unpacking libnetfilter-queue1 (1.0.2-2) ...</span><br><span class="line">Setting up libnetfilter-queue1 (1.0.2-2) ...</span><br><span class="line">Processing triggers for libc-bin (2.27-3ubuntu1) ...</span><br></pre></td></tr></table></figure><a id="more"></a><p>After install this package, I run the command again, this time it works<br>The bettercap is installed successfully.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~$ sudo bettercap -eval "caplets.update; ui.update; quit"</span><br><span class="line">bettercap v2.26.1 (built for linux amd64 with go1.13.3) [type 'help' for a list of commands]</span><br><span class="line"></span><br><span class="line">[21:01:54] [sys.log] [inf] caplets creating caplets install path /usr/local/share/bettercap/ ...</span><br><span class="line">[21:01:54] [sys.log] [inf] caplets downloading caplets from https://github.com/bettercap/caplets/archive/master.zip ...</span><br><span class="line">[21:01:55] [sys.log] [inf] caplets installing caplets to /usr/local/share/bettercap/caplets ...</span><br><span class="line">[21:01:55] [sys.log] [inf] ui checking latest stable release ...</span><br><span class="line">[21:01:55] [sys.log] [inf] ui downloading ui v1.3.0 from https://github.com/bettercap/ui/releases/download/v1.3.0/ui.zip ...</span><br><span class="line">[21:02:01] [sys.log] [inf] ui installing to /usr/local/share/bettercap/ui ...</span><br><span class="line">[21:02:02] [sys.log] [inf] ui installation complete, you can now run the http-ui (or https-ui) caplet to start the UI.</span><br></pre></td></tr></table></figure><h2 id="create-systemd-service-file-etc-systemd-system-bettercap-service"><a href="#create-systemd-service-file-etc-systemd-system-bettercap-service" class="headerlink" title="create systemd service file: /etc/systemd/system/bettercap.service"></a>create systemd service file: /etc/systemd/system/bettercap.service</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">[Unit]</span><br><span class="line">Description=bettercap api.rest service.</span><br><span class="line">Documentation=https://bettercap.org</span><br><span class="line">Wants=network.target</span><br><span class="line">After=pwngrid.service</span><br><span class="line"></span><br><span class="line">[Service]</span><br><span class="line">Type=simple</span><br><span class="line">PermissionsStartOnly=true</span><br><span class="line">ExecStart=/usr/bin/bettercap-launcher</span><br><span class="line">Restart=always</span><br><span class="line">RestartSec=30</span><br><span class="line"></span><br><span class="line">[Install]</span><br><span class="line">WantedBy=multi-user.target</span><br></pre></td></tr></table></figure><h2 id="copy-launch-file-to-usr-bin-bettercap-launcher"><a href="#copy-launch-file-to-usr-bin-bettercap-launcher" class="headerlink" title="copy launch file to /usr/bin/bettercap-launcher"></a>copy launch file to /usr/bin/bettercap-launcher</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">#!/usr/bin/env bash</span><br><span class="line">/usr/bin/monstart</span><br><span class="line">if [[ $(ifconfig | grep usb0 | grep RUNNING) ]] || [[ $(cat /sys/class/net/eth0/carrier) ]]; then</span><br><span class="line"> # if override file exists, go into auto mode</span><br><span class="line"> if [ -f /root/.pwnagotchi-auto ]; then</span><br><span class="line"> /usr/bin/bettercap -no-colors -caplet pwnagotchi-auto -iface mon0</span><br><span class="line"> else</span><br><span class="line"> /usr/bin/bettercap -no-colors -caplet pwnagotchi-manual -iface mon0</span><br><span class="line"> fi</span><br><span class="line">else</span><br><span class="line"> /usr/bin/bettercap -no-colors -caplet pwnagotchi-auto -iface mon0</span><br><span class="line">fi</span><br></pre></td></tr></table></figure><p>After these steps, I want to start bettercap servive<br>first I want to run bettercap-luncher</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">root@ubuntu1804:/usr/bin# ./bettercap-launcher </span><br><span class="line">./bettercap-launcher: line 2: /usr/bin/monstart: No such file or directory</span><br><span class="line">cat: /sys/class/net/eth0/carrier: No such file or directory</span><br><span class="line">bettercap v2.26.1 (built for linux amd64 with go1.13.3) [type 'help' for a list of commands]</span><br><span class="line"></span><br><span class="line">no interface matching 'mon0' found.</span><br></pre></td></tr></table></figure><p>google monstart, it said it is based on kali linux, not Ubuntu.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">root@ubuntu1804:/usr/bin# dpkg -l |grep kali</span><br><span class="line">root@ubuntu1804:/usr/bin# apt install monstart</span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">E: Unable to locate package monstart</span><br></pre></td></tr></table></figure><p>so monstart can not be install on Ubuntu.</p><p>try to start bettercap service, it is supposed to be failed.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">root@ubuntu1804:/usr/bin# systemctl enable bettercap</span><br><span class="line">Created symlink /etc/systemd/system/multi-user.target.wants/bettercap.service → /etc/systemd/system/bettercap.service.</span><br><span class="line">root@ubuntu1804:/usr/bin# systemctl start bettercap</span><br><span class="line">root@ubuntu1804:/usr/bin# systemctl status bettercap</span><br><span class="line">● bettercap.service - bettercap api.rest service.</span><br><span class="line"> Loaded: loaded (/etc/systemd/system/bettercap.service; enabled; vendor preset: enabled)</span><br><span class="line"> Active: activating (auto-restart) (Result: exit-code) since Fri 2019-11-22 12:38:59 PST; 4s ago</span><br><span class="line"> Docs: https://bettercap.org</span><br><span class="line"> Process: 25242 ExecStart=/usr/bin/bettercap-launcher (code=exited, status=203/EXEC)</span><br><span class="line"> Main PID: 25242 (code=exited, status=203/EXEC)</span><br><span class="line"></span><br><span class="line">Nov 22 12:38:59 ubuntu1804 systemd[1]: bettercap.service: Failed with result 'exit-code'.</span><br></pre></td></tr></table></figure><p>Conclusion: can not run bettercap on local Ubuntu since some command missing.</p>]]></content>
<tags>
<tag> bettercap </tag>
</tags>
</entry>
<entry>
<title>Pwnagotchi: using docker image to install bettercap</title>
<link href="/2019/11/22/Pwnagotchi-using-docker-image/"/>
<url>/2019/11/22/Pwnagotchi-using-docker-image/</url>
<content type="html"><![CDATA[<p>This time, I try to install bettercap docker image on raspberry Pi3B+.</p><h2 id="Install-dependency-package-first"><a href="#Install-dependency-package-first" class="headerlink" title="Install dependency package first"></a>Install dependency package first</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# apt install libnetfilter-queue1</span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">The following package was automatically installed and is no longer required:</span><br><span class="line"> point-rpi</span><br><span class="line">Use 'apt autoremove' to remove it.</span><br><span class="line">The following NEW packages will be installed:</span><br><span class="line"> libnetfilter-queue1</span><br><span class="line">0 upgraded, 1 newly installed, 0 to remove and 1 not upgraded.</span><br><span class="line">Need to get 10.7 kB of archives.</span><br><span class="line">After this operation, 36.9 kB of additional disk space will be used.</span><br><span class="line">Get:1 http://mirrors.ocf.berkeley.edu/raspbian/raspbian buster/main armhf libnetfilter-queue1 armhf 1.0.3-1 [10.7 kB]</span><br><span class="line">Fetched 10.7 kB in 1s (12.1 kB/s) </span><br><span class="line">Selecting previously unselected package libnetfilter-queue1.</span><br><span class="line">(Reading database ... 154638 files and directories currently installed.)</span><br><span class="line">Preparing to unpack .../libnetfilter-queue1_1.0.3-1_armhf.deb ...</span><br><span class="line">Unpacking libnetfilter-queue1 (1.0.3-1) ...</span><br><span class="line">Setting up libnetfilter-queue1 (1.0.3-1) ...</span><br><span class="line">Processing triggers for libc-bin (2.28-10+rpi1) ...</span><br><span class="line">root@treehouses:~# apt show libnetfilter-queue1</span><br><span class="line"></span><br><span class="line">Package: libnetfilter-queue1</span><br><span class="line">Version: 1.0.3-1</span><br><span class="line">Priority: optional</span><br><span class="line">Section: libs</span><br><span class="line">Source: libnetfilter-queue</span><br><span class="line">Maintainer: Debian Netfilter Packaging Team <[email protected]></span><br><span class="line">Installed-Size: 36.9 kB</span><br><span class="line">Depends: libc6 (>= 2.4), libmnl0 (>= 1.0.3-4~), libnfnetlink0</span><br><span class="line">Homepage: http://www.netfilter.org/projects/libnetfilter_queue/</span><br><span class="line">Download-Size: 10.7 kB</span><br><span class="line">APT-Manual-Installed: yes</span><br><span class="line">APT-Sources: http://raspbian.raspberrypi.org/raspbian buster/main armhf Packages</span><br><span class="line">Description: Netfilter netlink-queue library</span><br><span class="line"> libnetfilter_queue is a userspace library providing an API to packets</span><br><span class="line"> that have been queued by the kernel packet filter. It is part of a</span><br><span class="line"> system that deprecates the old ip_queue / libipq mechanism.</span><br></pre></td></tr></table></figure><h2 id="Install-bettercap-docker-image"><a href="#Install-bettercap-docker-image" class="headerlink" title="Install bettercap docker image"></a>Install bettercap docker image</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# docker pull bettercap/bettercap</span><br><span class="line">Using default tag: latest</span><br><span class="line">latest: Pulling from bettercap/bettercap</span><br><span class="line">bdf0201b3a05: Pull complete </span><br><span class="line">1465d5cbc6a8: Pull complete </span><br><span class="line">59da5739fafc: Pull complete </span><br><span class="line">86a51d61314d: Pull complete </span><br><span class="line">544433dabf48: Pull complete </span><br><span class="line">Digest: sha256:c7497e0839238a0a0d4920e583e765d7b53f794dea70f85882baa42c06ad8cbd</span><br><span class="line">Status: Downloaded newer image for bettercap/bettercap:latest</span><br><span class="line">docker.io/bettercap/bettercap:latest</span><br><span class="line">root@treehouses:~# docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">bettercap/bettercap latest c921a193dd94 6 months ago 51.1MB</span><br></pre></td></tr></table></figure><a id="more"></a><h2 id="pull-latest-source-code-build-of-the-image"><a href="#pull-latest-source-code-build-of-the-image" class="headerlink" title="pull latest source code build of the image"></a>pull latest source code build of the image</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# docker pull bettercap/dev</span><br><span class="line">Using default tag: latest</span><br><span class="line">latest: Pulling from bettercap/dev</span><br><span class="line">89d9c30c1d48: Pull complete </span><br><span class="line">cb311b5b246f: Pull complete </span><br><span class="line">76358ea99b60: Pull complete </span><br><span class="line">2d8598aa817c: Pull complete </span><br><span class="line">7eb6779e4cc2: Pull complete </span><br><span class="line">Digest: sha256:d7fe702b51a451349e10783e614e127330c8aa78eb2b7bb651fba289def280ad</span><br><span class="line">Status: Downloaded newer image for bettercap/dev:latest</span><br><span class="line">docker.io/bettercap/dev:latest</span><br><span class="line">root@treehouses:~# docker image ls</span><br><span class="line">REPOSITORY TAG IMAGE ID CREATED SIZE</span><br><span class="line">bettercap/dev latest ff15dd5f24e3 9 days ago 50.8MB</span><br><span class="line">bettercap/bettercap latest c921a193dd94 6 months ago 51.1MB</span><br></pre></td></tr></table></figure><h2 id="run-bettercap-image"><a href="#run-bettercap-image" class="headerlink" title="run bettercap image"></a>run bettercap image</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">root@treehouses:~# docker run -it --privileged --net=host bettercap/bettercap -h bettercap</span><br><span class="line">standard_init_linux.go:211: exec user process caused "exec format error"</span><br></pre></td></tr></table></figure><p>it shows failed.<br>I found this image is built for x86_64, I can’t run it on Pi(ARM).</p>]]></content>
<tags>
<tag> bettercap </tag>
<tag> Docker </tag>
</tags>
</entry>
<entry>
<title>Pwnagotchi project: Using Pwnagotchi image</title>
<link href="/2019/11/19/Pwnagotchi-project/"/>
<url>/2019/11/19/Pwnagotchi-project/</url>
<content type="html"><![CDATA[<h2 id="what-is-Pwnagotchi"><a href="#what-is-Pwnagotchi" class="headerlink" title="what is Pwnagotchi"></a>what is Pwnagotchi</h2><p>Pwnagotchi is an A2C-based “AI” powered by bettercap that learns from its surrounding WiFi environment in order to maximize the crackable WPA key material it captures (either through passive sniffing or by performing deauthentication and association attacks). This material is collected on disk as PCAP files containing any form of crackable handshake supported by hashcat, including full and half WPA handshakes as well as PMKIDs.</p><h2 id="falshing-an-image"><a href="#falshing-an-image" class="headerlink" title="falshing an image"></a>falshing an image</h2><p>download the <a href="https://github.com/evilsocket/pwnagotchi/releases" target="_blank" rel="noopener">the latest image</a><br>then unzip it into present directory</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~/Downloads$ dd if=pwnagotchi-raspbian-lite-v1.3.0.img of=/dev/sdcard</span><br></pre></td></tr></table></figure><h2 id="config-file"><a href="#config-file" class="headerlink" title="config file"></a>config file</h2><p>before boot this image on RPI 3B+, I need to config it first. I mount this device to /mnt/1.<br>then add config.yml to it.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">main:</span><br><span class="line"> name: 'pwnagotchi'</span><br><span class="line"> whitelist:</span><br><span class="line"> - 'YourHomeNetworkMaybe'</span><br><span class="line"> plugins:</span><br><span class="line"> grid:</span><br><span class="line"> enabled: true</span><br><span class="line"> report: true</span><br><span class="line"> exclude:</span><br><span class="line"> - 'YourHomeNetworkMaybe'</span><br></pre></td></tr></table></figure><p>Because I dont have RPi 0w, so I try to use web UI (instead of an e-ink display attached to RPi0W) to see your Pwnagotchi’s face.</p><p>I add these to my config.yml file</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">ui:</span><br><span class="line"> web:</span><br><span class="line"> username: my_new_username</span><br><span class="line"> password: my_new_password</span><br></pre></td></tr></table></figure><p>In order to reduce power requirements I can lower cpu frequency (underclocking). Edit my /boot/config.txt and uncomment the arm_freq=800.</p><h2 id="First-boot"><a href="#First-boot" class="headerlink" title="First boot"></a>First boot</h2><p>It shows like following.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br></pre></td><td class="code"><pre><span class="line">Linux pwnagotchi 4.19.81-Re4son-v7+ #1 SMP Wed Nov 6 10:16:47 AEDT 2019 armv7l</span><br><span class="line">(◕‿‿◕) pwnagotchi</span><br><span class="line"></span><br><span class="line">Hi! I'm a pwnagotchi, please take good care of me!</span><br><span class="line">Here are some basic things you need to know to raise me properly!</span><br><span class="line"></span><br><span class="line">If you want to change my configuration, use /etc/pwnagotchi/config.yml</span><br><span class="line"></span><br><span class="line">All the configuration options can be found on /etc/pwnagotchi/default.yml,</span><br><span class="line">but don't change this file because I will recreate it every time I'm restarted!</span><br><span class="line"></span><br><span class="line">I'm managed by systemd. Here are some basic commands.</span><br><span class="line"></span><br><span class="line">If you want to know what I'm doing, you can check my logs with the command</span><br><span class="line">journalctl -fu pwnagotchi</span><br><span class="line"></span><br><span class="line">If you want to know if I'm running, you can use</span><br><span class="line">systemctl status pwnagotchi</span><br><span class="line"></span><br><span class="line">You can restart me using</span><br><span class="line">systemctl restart pwnagotchi</span><br><span class="line"></span><br><span class="line">But be aware I will go into MANUAL mode when restarted!</span><br><span class="line">You can put me back into AUTO mode using</span><br><span class="line">touch /root/.pwnagotchi-auto && systemctl restart pwnagotchi</span><br><span class="line"></span><br><span class="line">You learn more about me at https://pwnagotchi.ai/</span><br><span class="line">Last login: Wed Jul 10 01:30:38 2019 from 192.168.0.26</span><br><span class="line"></span><br><span class="line">SSH is enabled and the default password for the 'pi' user has not been changed.</span><br><span class="line">This is a security risk - please login as the 'pi' user and type 'passwd' to set a new password.</span><br></pre></td></tr></table></figure><p>check the service </p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">pi@pwnagotchi:~ $ systemctl status pwnagotchi.service </span><br><span class="line">● pwnagotchi.service - pwnagotchi Deep Reinforcement Learning instrumenting bettercap for WiFI pwning.</span><br><span class="line"> Loaded: loaded (/etc/systemd/system/pwnagotchi.service; enabled; vendor preset: enabled)</span><br><span class="line"> Active: active (running) since Wed 2019-07-10 02:17:04 BST; 2min 45s ago</span><br><span class="line"> Docs: https://pwnagotchi.ai</span><br><span class="line"> Main PID: 406 (bash)</span><br><span class="line"> Tasks: 24 (limit: 2319)</span><br><span class="line"> CGroup: /system.slice/pwnagotchi.service</span><br><span class="line"> ├─406 bash /usr/bin/pwnagotchi-launcher</span><br><span class="line"> ├─468 /usr/bin/python3 /usr/local/bin/pwnagotchi</span><br><span class="line"> └─811 orted --hnp --set-sid --report-uri 14 --singleton-died-pipe 15 -mca state_novm_select 1 -mca ess hnp -mca pmix ^s1,s2,cray,is</span><br><span class="line"></span><br><span class="line">Jul 10 02:18:06 pwnagotchi pwnagotchi-launcher[406]: Instructions for updating:</span><br><span class="line">Jul 10 02:18:06 pwnagotchi pwnagotchi-launcher[406]: Please use `layer.__call__` method instead.</span><br><span class="line">Jul 10 02:18:06 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:18:06,735] [WARNING] From /usr/local/lib/python3.7/dist-packages/tensorflo</span><br><span class="line">Jul 10 02:19:33 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:33,331] [WARNING] From /usr/local/lib/python3.7/dist-packages/stable_ba</span><br><span class="line">Jul 10 02:19:36 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:36,093] [ERROR] got data on channel 149, we can store 140 channels</span><br><span class="line">Jul 10 02:19:36 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:36,112] [ERROR] got data on channel 149, we can store 140 channels</span><br><span class="line">Jul 10 02:19:36 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:36,116] [ERROR] got data on channel 149, we can store 140 channels</span><br><span class="line">Jul 10 02:19:36 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:36,118] [ERROR] got data on channel 149, we can store 140 channels</span><br><span class="line">Jul 10 02:19:36 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:36,129] [ERROR] got data on channel 149, we can store 140 channels</span><br><span class="line">Jul 10 02:19:41 pwnagotchi pwnagotchi-launcher[406]: [2019-07-10 02:19:41,825] [INFO] sending association frame to clarkwifi (b0:2a:43:e6:6f:f</span><br></pre></td></tr></table></figure><p>all the handshake eaten by pwnagotchi can be found under /root/handshake</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">root@pwnagotchi:~/handshakes# ls -al</span><br><span class="line">total 52</span><br><span class="line">drwxr-xr-x 2 root root 4096 Jul 10 02:28 .</span><br><span class="line">drwx------ 9 root root 4096 Jul 10 02:17 ..</span><br><span class="line">-rw-r--r-- 1 root root 1780 Jul 10 02:28 abfguest_d66e0e3131a4.pcap</span><br><span class="line">-rw-r--r-- 1 root root 4824 Jul 10 02:20 ATTtpcaygs_f82dc0d869e0.pcap</span><br><span class="line">-rw-r--r-- 1 root root 6264 Jul 10 02:21 ATTvmtPDGs_2c9569519550.pcap</span><br><span class="line">-rw-r--r-- 1 root root 2544 Jul 10 02:22 DIRECT19HPOfficeJet3830_10e7c694ba1a.pcap</span><br><span class="line">-rw-r--r-- 1 root root 1892 Jul 10 02:25 Hailey716_b02a43ec8ab0.pcap</span><br><span class="line">-rw-r--r-- 1 root root 2812 Jul 10 02:23 hidden_963badcbb2be.pcap</span><br><span class="line">-rw-r--r-- 1 root root 2476 Jul 10 02:23 NETGEARORBIhidden86_963badcbb2be.pcap</span><br><span class="line">-rw-r--r-- 1 root root 2283 Jul 10 02:27 ngHub319444NG01912_dcef09d5a816.pcap</span><br><span class="line">-rw-r--r-- 1 root root 2484 Jul 10 02:21 PeakyBlinders24G_9c3dcf98b8b3.pcap</span><br></pre></td></tr></table></figure><img src="/2019/11/19/Pwnagotchi-project/image_login.png" class=""><img src="/2019/11/19/Pwnagotchi-project/make_0_friend.png" class=""><img src="/2019/11/19/Pwnagotchi-project/bettercap.png" class="">]]></content>
<tags>
<tag> pwnagotchi - </tag>
<tag> wifi </tag>
</tags>
</entry>
<entry>
<title>npm analyzer project: part 5</title>
<link href="/2019/11/07/npm-analyzer-project-part-5/"/>
<url>/2019/11/07/npm-analyzer-project-part-5/</url>
<content type="html"><![CDATA[<h2 id="config-the-npm-file"><a href="#config-the-npm-file" class="headerlink" title="config the npm file."></a>config the npm file.</h2><p>This project uses npm config for configuration. I need to create config/local.json5 file to override the configuration as necessary, especially to define githubTokens. After working a lot, I found I need to clone the npm-analyzer repo actuallya. I do not know whether the work before is worthable if I already clone the whole repo.</p><h4 id="clone-the-github-npm-analyzer"><a href="#clone-the-github-npm-analyzer" class="headerlink" title="clone the github npm-analyzer"></a>clone the github npm-analyzer</h4><p>under my npm_analyzer dirctory, clone the repo to my local machine.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git clone https://github.com/npms-io/npms-analyzer.git</span><br></pre></td></tr></table></figure><h4 id="copy-the-default-json-file-to-create-my-own-local-json5-file"><a href="#copy-the-default-json-file-to-create-my-own-local-json5-file" class="headerlink" title="copy the default.json file to create my own local.json5 file"></a>copy the default.json file to create my own local.json5 file</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ cp default.json local.json5</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer/npms-analyzer/config$ ls -al</span><br><span class="line">total 24</span><br><span class="line">drwxr-xr-x 4 vagrant vagrant 4096 Nov 7 07:19 .</span><br><span class="line">drwxr-xr-x 9 vagrant vagrant 4096 Nov 7 05:38 ..</span><br><span class="line">drwxr-xr-x 2 vagrant vagrant 4096 Nov 7 05:31 couchdb</span><br><span class="line">-rw-r--r-- 1 vagrant vagrant 1187 Nov 7 05:31 default.json5</span><br><span class="line">drwxr-xr-x 2 vagrant vagrant 4096 Nov 7 05:31 elasticsearch</span><br><span class="line">-rw-r--r-- 1 vagrant vagrant 116 Nov 7 05:47 local.json5</span><br></pre></td></tr></table></figure><h4 id="create-my-githubtoken-write-it-to-local-json5-file"><a href="#create-my-githubtoken-write-it-to-local-json5-file" class="headerlink" title="create my githubtoken, write it to local.json5 file"></a>create my githubtoken, write it to local.json5 file</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> // Github tokens to be used by token-dealer</span><br><span class="line"> githubTokens: ['227dc6ab8270d13f5ac2134c466'],</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h4 id="check-default-json-file"><a href="#check-default-json-file" class="headerlink" title="check default.json file"></a>check default.json file</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> // Databases & similar stuff</span><br><span class="line"> couchdbNpm: {</span><br><span class="line"> url: 'http://admin:[email protected]:5984/npm',</span><br><span class="line"> requestDefaults: { timeout: 15000 },</span><br><span class="line"> },</span><br><span class="line"> couchdbNpms: {</span><br><span class="line"> url: 'http://admin:[email protected]:5984/npms',</span><br><span class="line"> requestDefaults: { timeout: 15000 },</span><br><span class="line"> },</span><br><span class="line"> elasticsearch: {</span><br><span class="line"> host: 'http://127.0.0.1:9200',</span><br><span class="line"> requestTimeout: 15000,</span><br><span class="line"> apiVersion: '6.3',</span><br><span class="line"> log: null,</span><br><span class="line"> },</span><br><span class="line"> queue: {</span><br><span class="line"> name: 'npms',</span><br><span class="line"> addr: 'amqp://guest:[email protected]',</span><br><span class="line"> options: { maxPriority: 1 },</span><br><span class="line"> },</span><br><span class="line"></span><br><span class="line"> // List of packages that will be ignored by the CLI consume command (analysis process)</span><br><span class="line"> blacklist: {</span><br><span class="line"> 'hownpm': 'Invalid version: 1.01',</span><br><span class="line"> 'zachtestproject1': 'Test project that makes registry return 500 internal',</span><br><span class="line"> 'zachtestproject2': 'Test project that makes registry return 500 internal',</span><br><span class="line"> 'zachtestproject3': 'Test project that makes registry return 500 internal',</span><br><span class="line"> 'zachtestproject4': 'Test project that makes registry return 500 internal',</span><br><span class="line"> 'broken-package-truncated-tar-header': 'Broken tarball',</span><br><span class="line"> },</span><br><span class="line"></span><br><span class="line"> // Github tokens to be used by token-dealer</span><br><span class="line"> githubTokens: [],</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>after checking default.json file, I found there are two databases, one is couchdbNpms which is for queue, and another one is couchdbNpm which is for replication from <a href="https://replicate.npmjs.com/registry" target="_blank" rel="noopener">https://replicate.npmjs.com/registry</a>. I only created npms, so I need to go back couchdb to create npm database and redo the replication work. It takes long time to replication the couchdb and also shows “error” for replication funtion on couchdb website.</p><a id="more"></a><p>Note:<br>Since I am working my project on cloud, I need to use ssh tunnel to log in http website to work on couchdb and rabbitmq and elasticsearch.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">ssh -L localhost:5984:localhost:5984 ubuntu@ipaddress # open couchdb</span><br><span class="line">ssh -L localhost:8080:localhost:15672 ubuntu@ipaddress # open rabbitMQ</span><br><span class="line">ssh -L localhost:9100:localhost:9100 ubuntu@ipaddress # open elasticsearch</span><br><span class="line">ssh -L localhost:9200:localhost:9200 ubuntu@ipaddress # open elasticsearch</span><br></pre></td></tr></table></figure><p>after tunnel, it will show like this</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~/anna_tech_blog/newblog/source/_posts$ ssh -L localhost:8080:localhost:15672 ubuntu@ip address</span><br><span class="line">Welcome to Ubuntu 18.04.3 LTS (GNU/Linux 4.15.0-1026-oracle x86_64)</span><br><span class="line"></span><br><span class="line"> * Documentation: https://help.ubuntu.com</span><br><span class="line"> * Management: https://landscape.canonical.com</span><br><span class="line"> * Support: https://ubuntu.com/advantage</span><br><span class="line"></span><br><span class="line"> System information disabled due to load higher than 2.0</span><br><span class="line"></span><br><span class="line"> * Kata Containers are now fully integrated in Charmed Kubernetes 1.16!</span><br><span class="line"> Yes, charms take the Krazy out of K8s Kata Kluster Konstruction.</span><br><span class="line"></span><br><span class="line"> https://ubuntu.com/kubernetes/docs/release-notes</span><br><span class="line"></span><br><span class="line"> * Canonical Livepatch is available for installation.</span><br><span class="line"> - Reduce system reboots and improve kernel security. Activate at:</span><br><span class="line"> https://ubuntu.com/livepatch</span><br><span class="line"></span><br><span class="line">55 packages can be updated.</span><br><span class="line">0 updates are security updates.</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">*** System restart required ***</span><br><span class="line">Last login: Tue Nov 19 19:47:26 2019 from ip address</span><br></pre></td></tr></table></figure><p>login </p><img src="/2019/11/07/npm-analyzer-project-part-5/login.png" class=""><p>database</p><img src="/2019/11/07/npm-analyzer-project-part-5/database.png" class=""><p>active task</p><img src="/2019/11/07/npm-analyzer-project-part-5/activetask.png" class=""><p>There are something wrong with replication, so the verification shows error</p><img src="/2019/11/07/npm-analyzer-project-part-5/verify.png" class=""><p>then login the rabbtMQ</p><img src="/2019/11/07/npm-analyzer-project-part-5/rabbitmq.png" class=""><p>login the elasticsearch</p><img src="/2019/11/07/npm-analyzer-project-part-5/elastic2.png" class=""><img src="/2019/11/07/npm-analyzer-project-part-5/elastic1.png" class=""><h2 id="npms-analyzer-observe"><a href="#npms-analyzer-observe" class="headerlink" title="npms-analyzer observe"></a>npms-analyzer observe</h2><p>The observe command starts observing changes that occur in the npm registry as well as packages that were not analyzed for a while. Each reported package will be pushed into a queue to be processed by the queue consumers.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer/npms-analyzer$ ./cli.js observe --log-level debug | pino-pretty</span><br><span class="line">[1574225327914] INFO (npms-analyzer/31902 on cli): RabbitMQ connect and setup done successfully</span><br><span class="line"> module: "queue"</span><br><span class="line">[1574225327942] DEBUG (npms-analyzer/31902 on cli): Queue is ready</span><br><span class="line"> module: "bootstrap"</span><br><span class="line">[1574225334791] DEBUG (npms-analyzer/31902 on cli): CouchDB for npm is ready</span><br><span class="line"> module: "bootstrap"</span><br><span class="line">[1574225338107] DEBUG (npms-analyzer/31902 on cli): CouchDB for npms is ready</span><br><span class="line"> module: "bootstrap"</span><br><span class="line">[1574225338118] INFO (npms-analyzer/31902 on cli): Starting realtime observer..</span><br><span class="line"> module: "observer/realtime"</span><br><span class="line">[1574225338135] INFO (npms-analyzer/31902 on cli): Starting stale observer..</span><br><span class="line"> module: "observer/stale"</span><br><span class="line">[1574225338149] DEBUG (npms-analyzer/31902 on cli): Looking for stale packages (failed)..</span><br><span class="line"> module: "observer/stale"</span><br><span class="line">[1574225343426] INFO (npms-analyzer/31902 on cli): Will now start to follow changes since 513900-g1AAAAI7eJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOYmBkXF9LlCM3TIl2STV0hJdPQ4TkhSAZJI9zBCGf0vBhhgZWSaZJqUQa4gDyJB4uCF_2cGGmKQYmlkYGBNrSALIkHq4Ib-yIS4xNDVKtTQk0pA8FiDJ0ACkgObMhxj04zzENYamppbmpBm0AGLQfqhBjWCDDI3MjC1NzEky6ADEoPtQgy6CDTI2ME5MMTMgyaAHEIOgYfS1FWyQuYlpqqFZIrq2LAD-NLFU</span><br><span class="line"> module: "observer/realtime"</span><br><span class="line"> since: "513900-g1AAAAI7eJzLYWBg4MhgTmHgzcvPy09JdcjLz8gvLskBCjMlMiTJ____PyuDOYmBkXF9LlCM3TIl2STV0hJdPQ4TkhSAZJI9zBCGf0vBhhgZWSaZJqUQa4gDyJB4uCF_2cGGmKQYmlkYGBNrSALIkHq4Ib-yIS4xNDVKtTQk0pA8FiDJ0ACkgObMhxj04zzENYamppbmpBm0AGLQfqhBjWCDDI3MjC1NzEky6ADEoPtQgy6CDTI2ME5MMTMgyaAHEIOgYfS1FWyQuYlpqqFZIrq2LAD-NLFU"</span><br><span class="line">[1574225348422] DEBUG (npms-analyzer/31902 on cli): Flushing changes buffer with a total of 250 changes..</span><br><span class="line"> module: "observer/realtime"</span><br><span class="line">[1574225353118] INFO (npms-analyzer/31902 on cli): pid: 31902; memory: 24.00 MB; uptime: 51 seconds</span><br><span class="line"> module: "stats/process"</span><br><span class="line">[1574225353213] ERROR (npms-analyzer/31902 on cli): Stale observer failed when notifying stale packages (failed)</span><br><span class="line"> module: "observer/stale"</span><br><span class="line"> err: {</span><br><span class="line"> "type": "Error",</span><br><span class="line"> "message": "error happened in your connection",</span><br><span class="line"> "stack":</span><br><span class="line"> Error: error happened in your connection</span><br><span class="line"> at ClientRequest.<anonymous> (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/request/request.js:816:19)</span><br><span class="line"> at Object.onceWrapper (events.js:286:20)</span><br><span class="line"> at ClientRequest.emit (events.js:198:13)</span><br><span class="line"> at Socket.emitRequestTimeout (_http_client.js:662:40)</span><br><span class="line"> at Object.onceWrapper (events.js:286:20)</span><br><span class="line"> at Socket.emit (events.js:198:13)</span><br><span class="line"> at Socket._onTimeout (net.js:442:8)</span><br><span class="line"> at ontimeout (timers.js:436:11)</span><br><span class="line"> at tryOnTimeout (timers.js:300:5)</span><br><span class="line"> at listOnTimeout (timers.js:263:5)</span><br><span class="line"> at Timer.processTimers (timers.js:223:10)</span><br><span class="line"> From previous event:</span><br><span class="line"> at RowsReaderStream._queryFn (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/couchdb-iterator/index.js:48:16)</span><br><span class="line"> at RowsReaderStream._read (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/couchdb-iterator/lib/rowsReader.js:23:14)</span><br><span class="line"> at RowsReaderStream.Readable.read (_stream_readable.js:470:10)</span><br><span class="line"> at resume_ (_stream_readable.js:949:12)</span><br><span class="line"> at process._tickCallback (internal/process/next_tick.js:63:19)</span><br><span class="line"> "name": "Error",</span><br><span class="line"> "scope": "socket",</span><br><span class="line"> "errid": "request",</span><br><span class="line"> "code": "ESOCKETTIMEDOUT",</span><br><span class="line"> "connect": false,</span><br><span class="line"> "description": "ESOCKETTIMEDOUT",</span><br><span class="line"> "stacktrace": [</span><br><span class="line"> "Error: ESOCKETTIMEDOUT",</span><br><span class="line"> " at ClientRequest.<anonymous> (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/request/request.js:816:19)",</span><br><span class="line"> " at Object.onceWrapper (events.js:286:20)",</span><br><span class="line"> " at ClientRequest.emit (events.js:198:13)",</span><br><span class="line"> " at Socket.emitRequestTimeout (_http_client.js:662:40)",</span><br><span class="line"> " at Object.onceWrapper (events.js:286:20)",</span><br><span class="line"> " at Socket.emit (events.js:198:13)",</span><br><span class="line"> " at Socket._onTimeout (net.js:442:8)",</span><br><span class="line"> " at ontimeout (timers.js:436:11)",</span><br><span class="line"> " at tryOnTimeout (timers.js:300:5)",</span><br><span class="line"> " at listOnTimeout (timers.js:263:5)",</span><br><span class="line"> " at Timer.processTimers (timers.js:223:10)"</span><br><span class="line"> ]</span><br><span class="line"> }</span><br><span class="line">[1574225353248] DEBUG (npms-analyzer/31902 on cli): Looking for stale packages (normal)..</span><br><span class="line"> module: "observer/stale"</span><br><span class="line">[1574225353270] INFO (npms-analyzer/31902 on cli): Queue stat</span><br><span class="line"> module: "stats/queue"</span><br><span class="line"> stat: {</span><br><span class="line"> "messageCount": 498961,</span><br><span class="line"> "consumerCount": 0</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><p>it shows queue first successed, but failed after for some reason.</p><p>error message</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br></pre></td><td class="code"><pre><span class="line">[1574225383774] INFO (npms-analyzer/31902 on cli): Starting realtime observer..</span><br><span class="line"> module: "observer/realtime"</span><br><span class="line">[1574225383786] ERROR (npms-analyzer/31902 on cli): Realtime observer failed, restarting in a few moments..</span><br><span class="line"> module: "observer/realtime"</span><br><span class="line"> err: {</span><br><span class="line"> "type": "OperationalError",</span><br><span class="line"> "message": "error happened in your connection",</span><br><span class="line"> "stack":</span><br><span class="line"> Error: error happened in your connection</span><br><span class="line"> at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)</span><br><span class="line"> From previous event:</span><br><span class="line"> at RealtimeObserver._fetchLastSeq (/home/vagrant/npm_analyzer/npms-analyzer/lib/observers/realtime.js:123:31)</span><br><span class="line"> at RealtimeObserver._start (/home/vagrant/npm_analyzer/npms-analyzer/lib/observers/realtime.js:79:14)</span><br><span class="line"> at Timeout._restartTimeout.setTimeout [as _onTimeout] (/home/vagrant/npm_analyzer/npms-analyzer/lib/observers/realtime.js:89:22)</span><br><span class="line"> at ontimeout (timers.js:436:11)</span><br><span class="line"> at tryOnTimeout (timers.js:300:5)</span><br><span class="line"> at listOnTimeout (timers.js:263:5)</span><br><span class="line"> at Timer.processTimers (timers.js:223:10)</span><br><span class="line"> "cause": {</span><br><span class="line"> "type": "Error",</span><br><span class="line"> "message": "error happened in your connection",</span><br><span class="line"> "stack":</span><br><span class="line"> Error: connect ECONNREFUSED 127.0.0.1:5984</span><br><span class="line"> at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)</span><br><span class="line"> "name": "Error",</span><br><span class="line"> "scope": "socket",</span><br><span class="line"> "errid": "request",</span><br><span class="line"> "errno": "ECONNREFUSED",</span><br><span class="line"> "code": "ECONNREFUSED",</span><br><span class="line"> "syscall": "connect",</span><br><span class="line"> "address": "127.0.0.1",</span><br><span class="line"> "port": 5984,</span><br><span class="line"> "description": "connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> "stacktrace": [</span><br><span class="line"> "Error: connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> " at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)"</span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> "isOperational": true,</span><br><span class="line"> "scope": "socket",</span><br><span class="line"> "errid": "request",</span><br><span class="line"> "errno": "ECONNREFUSED",</span><br><span class="line"> "code": "ECONNREFUSED",</span><br><span class="line"> "syscall": "connect",</span><br><span class="line"> "address": "127.0.0.1",</span><br><span class="line"> "port": 5984,</span><br><span class="line"> "description": "connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> "stacktrace": [</span><br><span class="line"> "Error: connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> " at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)"</span><br><span class="line"> ]</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><h2 id="npms-analyzer-consume"><a href="#npms-analyzer-consume" class="headerlink" title="npms-analyzer consume"></a>npms-analyzer consume</h2><p>The consume command starts consuming the queue, running the analysis process for each queued package.</p><p>Since the first step is failed, this step also failed.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer/npms-analyzer$ ./cli.js consume --log-level debug --concurrency 5 | pino-pretty</span><br><span class="line">[1574225672854] WARN (npms-analyzer/32491 on cli): Check of npm failed</span><br><span class="line"> module: "bootstrap"</span><br><span class="line"> err: {</span><br><span class="line"> "type": "OperationalError",</span><br><span class="line"> "message": "error happened in your connection",</span><br><span class="line"> "stack":</span><br><span class="line"> Error: error happened in your connection</span><br><span class="line"> at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)</span><br><span class="line"> From previous event:</span><br><span class="line"> at promiseRetry (/home/vagrant/npm_analyzer/npms-analyzer/cmd/util/bootstrap.js:68:20)</span><br><span class="line"> at /home/vagrant/npm_analyzer/npms-analyzer/node_modules/promise-retry/index.js:29:24</span><br><span class="line"> at runCallback (timers.js:705:18)</span><br><span class="line"> at tryOnImmediate (timers.js:676:5)</span><br><span class="line"> at processImmediate (timers.js:658:5)</span><br><span class="line"> From previous event:</span><br><span class="line"> at RetryOperation._fn (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/promise-retry/index.js:28:14)</span><br><span class="line"> at RetryOperation.attempt (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/retry/lib/retry_operation.js:97:8)</span><br><span class="line"> at /home/vagrant/npm_analyzer/npms-analyzer/node_modules/promise-retry/index.js:26:19</span><br><span class="line"> From previous event:</span><br><span class="line"> at promiseRetry (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/promise-retry/index.js:25:12)</span><br><span class="line"> at bootstrapCouchdb (/home/vagrant/npm_analyzer/npms-analyzer/cmd/util/bootstrap.js:67:12)</span><br><span class="line"> at Promise.map (/home/vagrant/npm_analyzer/npms-analyzer/cmd/util/bootstrap.js:37:20)</span><br><span class="line"> at runCallback (timers.js:705:18)</span><br><span class="line"> at tryOnImmediate (timers.js:676:5)</span><br><span class="line"> at processImmediate (timers.js:658:5)</span><br><span class="line"> From previous event:</span><br><span class="line"> at bootstrap (/home/vagrant/npm_analyzer/npms-analyzer/cmd/util/bootstrap.js:33:20)</span><br><span class="line"> at Object.exports.handler (/home/vagrant/npm_analyzer/npms-analyzer/cmd/consume.js:105:5)</span><br><span class="line"> at Object.runCommand (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/yargs/lib/command.js:238:44)</span><br><span class="line"> at Object.parseArgs [as _parseArgs] (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/yargs/yargs.js:1063:30)</span><br><span class="line"> at Object.get [as argv] (/home/vagrant/npm_analyzer/npms-analyzer/node_modules/yargs/yargs.js:1004:21)</span><br><span class="line"> at Object.<anonymous> (/home/vagrant/npm_analyzer/npms-analyzer/cli.js:32:1)</span><br><span class="line"> at Module._compile (internal/modules/cjs/loader.js:778:30)</span><br><span class="line"> at Object.Module._extensions..js (internal/modules/cjs/loader.js:789:10)</span><br><span class="line"> at Module.load (internal/modules/cjs/loader.js:653:32)</span><br><span class="line"> at tryModuleLoad (internal/modules/cjs/loader.js:593:12)</span><br><span class="line"> at Function.Module._load (internal/modules/cjs/loader.js:585:3)</span><br><span class="line"> at Function.Module.runMain (internal/modules/cjs/loader.js:831:12)</span><br><span class="line"> at startup (internal/bootstrap/node.js:283:19)</span><br><span class="line"> at bootstrapNodeJSCore (internal/bootstrap/node.js:622:3)</span><br><span class="line"> "cause": {</span><br><span class="line"> "type": "Error",</span><br><span class="line"> "message": "error happened in your connection",</span><br><span class="line"> "stack":</span><br><span class="line"> Error: connect ECONNREFUSED 127.0.0.1:5984</span><br><span class="line"> at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)</span><br><span class="line"> "name": "Error",</span><br><span class="line"> "scope": "socket",</span><br><span class="line"> "errid": "request",</span><br><span class="line"> "errno": "ECONNREFUSED",</span><br><span class="line"> "code": "ECONNREFUSED",</span><br><span class="line"> "syscall": "connect",</span><br><span class="line"> "address": "127.0.0.1",</span><br><span class="line"> "port": 5984,</span><br><span class="line"> "description": "connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> "stacktrace": [</span><br><span class="line"> "Error: connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> " at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)"</span><br><span class="line"> ]</span><br><span class="line"> },</span><br><span class="line"> "isOperational": true,</span><br><span class="line"> "scope": "socket",</span><br><span class="line"> "errid": "request",</span><br><span class="line"> "errno": "ECONNREFUSED",</span><br><span class="line"> "code": "ECONNREFUSED",</span><br><span class="line"> "syscall": "connect",</span><br><span class="line"> "address": "127.0.0.1",</span><br><span class="line"> "port": 5984,</span><br><span class="line"> "description": "connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> "stacktrace": [</span><br><span class="line"> "Error: connect ECONNREFUSED 127.0.0.1:5984",</span><br><span class="line"> " at TCPConnectWrap.afterConnect [as oncomplete] (net.js:1106:14)"</span><br><span class="line"> ]</span><br><span class="line"> }</span><br></pre></td></tr></table></figure><h2 id="npms-analyzer-scoring"><a href="#npms-analyzer-scoring" class="headerlink" title="npms-analyzer scoring"></a>npms-analyzer scoring</h2><p>The scoring command, continuously iterates over the analysis results and calculates a score for all the npm packages, storing its result in elasticsearch.</p><p>I will try to run the last command.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer/npms-analyzer$ ./cli.js scoring</span><br><span class="line">{"level":60,"time":1574226093326,"pid":803,"hostname":"cli","name":"npms-analyzer","module":"cli/scoring","err":{"msg":"[mapper_parsing_exception] Root mapping definition has unsupported parameters: [score : {dynamic=false, _all={enabled=false}, properties={evaluation={type=object, properties={popularity={type=object, properties={downloadsCount={index=false, type=double}, dependentsCount={index=false, type=double}, communityInterest={index=false, type=double}, downloadsAcceleration={index=false, type=double}}}, maintenance={type=object, properties={releasesFrequency={index=false, type=double}, commitsFrequency={index=false, type=double}, issuesDistribution={index=false, type=double}, openIssues={index=false, type=double}}}, quality={type=object, properties={tests={index=false, type=double}, branding={index=false, type=double}, health={index=false, type=double}, carefulness={index=false, type=double}}}}}, score={type=object, properties={final={type=double}, detail={type=object, properties={popularity={type=double}, maintenance={type=double}, quality={type=double}}}}}, package={type=object, properties={date={index=false, type=date}, maintainers={type=object, properties={email={normalizer=raw, type=keyword}, username={normalizer=raw, type=keyword}}}, keywords={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, raw={analyzer=raw, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, author={type=object, properties={name={normalizer=raw, type=keyword}, email={normalizer=raw, type=keyword}, url={index=false, type=text}, username={normalizer=raw, type=keyword}}}, scope={normalizer=raw, type=keyword}, name={type=text, fields={standard={analyzer=standard, type=text}, autocomplete={search_analyzer=package, analyzer=package_autocomplete, type=text}, english={analyzer=package_english, type=text}, raw={normalizer=raw, type=keyword}, english_aggressive={analyzer=package_english_aggressive, type=text}, autocomplete_highlight={search_analyzer=package, analyzer=package_autocomplete_highlight, type=text, index_options=offsets}, autocomplete_keyword={search_analyzer=package_autocomplete_keyword_search, analyzer=package_autocomplete_keyword, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, description={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, publisher={type=object, properties={email={index=false, type=text}, username={index=false, type=text}}}, links={type=object, properties={bugs={index=false, type=text}, npm={index=false, type=text}, repository={index=false, type=text}, homepage={index=false, type=text}}}, version={index=false, type=text}}}, flags={type=object, properties={unstable={type=boolean}, deprecated={type=text, index_options=docs}, insecure={type=integer}}}}}]","path":"/npms-1574226087671","query":{},"body":"{\"settings\":{\"number_of_shards\":1,\"number_of_replicas\":0,\"analysis\":{\"tokenizer\":{\"autocomplete\":{\"type\":\"edgeNGram\",\"min_gram\":\"1\",\"max_gram\":\"15\",\"token_chars\":[\"letter\",\"digit\"]}},\"filter\":{\"non_alfanum_to_space\":{\"type\":\"pattern_replace\",\"pattern\":\"(?i)[^a-z0-9]+\",\"replacement\":\" \"},\"split_word\":{\"type\":\"word_delimiter\",\"generate_word_parts\":true,\"generate_number_parts\":true,\"catenate_words\":false,\"catenate_numbers\":false,\"catenate_all\":false,\"split_on_case_change\":true,\"preserve_original\":true,\"split_on_numerics\":true,\"stem_english_possessive\":true},\"edge_ngram\":{\"type\":\"edgeNGram\",\"min_gram\":\"4\",\"max_gram\":\"15\"},\"autocomplete\":{\"type\":\"edgeNGram\",\"min_gram\":\"1\",\"max_gram\":\"15\"},\"unique_on_same_position\":{\"type\":\"unique\",\"only_on_same_position\":false}},\"analyzer\":{\"package\":{\"tokenizer\":\"standard\",\"filter\":[\"asciifolding\",\"split_word\",\"lowercase\",\"unique_on_same_position\"]},\"package_english\":{\"tokenizer\":\"standard\",\"filter\":[\"asciifolding\",\"split_word\",\"lowercase\",\"kstem\",\"unique_on_same_position\"]},\"package_english_aggressive\":{\"tokenizer\":\"standard\",\"filter\":[\"asciifolding\",\"split_word\",\"lowercase\",\"porter_stem\",\"unique_on_same_position\"]},\"package_edge_ngram\":{\"tokenizer\":\"standard\",\"filter\":[\"asciifolding\",\"split_word\",\"lowercase\",\"edge_ngram\",\"unique_on_same_position\"]},\"package_autocomplete\":{\"tokenizer\":\"standard\",\"filter\":[\"asciifolding\",\"split_word\",\"lowercase\",\"autocomplete\",\"unique_on_same_position\"]},\"package_autocomplete_keyword\":{\"tokenizer\":\"keyword\",\"filter\":[\"asciifolding\",\"non_alfanum_to_space\",\"lowercase\",\"autocomplete\",\"trim\",\"unique_on_same_position\"]},\"package_autocomplete_keyword_search\":{\"tokenizer\":\"keyword\",\"filter\":[\"asciifolding\",\"non_alfanum_to_space\",\"lowercase\",\"trim\"]},\"package_autocomplete_highlight\":{\"tokenizer\":\"autocomplete\",\"filter\":[\"asciifolding\",\"non_alfanum_to_space\",\"lowercase\",\"trim\"]},\"raw\":{\"tokenizer\":\"keyword\",\"filter\":[\"asciifolding\",\"lowercase\",\"trim\"]}},\"normalizer\":{\"raw\":{\"type\":\"custom\",\"filter\":[\"asciifolding\",\"lowercase\",\"trim\"]}}}},\"mappings\":{\"score\":{\"dynamic\":false,\"_all\":{\"enabled\":false},\"properties\":{\"package\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"text\",\"fields\":{\"standard\":{\"type\":\"text\",\"analyzer\":\"standard\"},\"english\":{\"type\":\"text\",\"analyzer\":\"package_english\"},\"english_aggressive\":{\"type\":\"text\",\"analyzer\":\"package_english_aggressive\"},\"edge_ngram\":{\"type\":\"text\",\"analyzer\":\"package_edge_ngram\",\"search_analyzer\":\"package\"},\"autocomplete\":{\"type\":\"text\",\"analyzer\":\"package_autocomplete\",\"search_analyzer\":\"package\"},\"autocomplete_highlight\":{\"type\":\"text\",\"analyzer\":\"package_autocomplete_highlight\",\"search_analyzer\":\"package\",\"index_options\":\"offsets\"},\"autocomplete_keyword\":{\"type\":\"text\",\"analyzer\":\"package_autocomplete_keyword\",\"search_analyzer\":\"package_autocomplete_keyword_search\"},\"raw\":{\"type\":\"keyword\",\"normalizer\":\"raw\"}}},\"description\":{\"type\":\"text\",\"fields\":{\"standard\":{\"type\":\"text\",\"analyzer\":\"standard\"},\"english\":{\"type\":\"text\",\"analyzer\":\"package_english\"},\"english_aggressive\":{\"type\":\"text\",\"analyzer\":\"package_english_aggressive\"},\"edge_ngram\":{\"type\":\"text\",\"analyzer\":\"package_edge_ngram\",\"search_analyzer\":\"package\"}}},\"keywords\":{\"type\":\"text\",\"fields\":{\"standard\":{\"type\":\"text\",\"analyzer\":\"standard\"},\"english\":{\"type\":\"text\",\"analyzer\":\"package_english\"},\"english_aggressive\":{\"type\":\"text\",\"analyzer\":\"package_english_aggressive\"},\"edge_ngram\":{\"type\":\"text\",\"analyzer\":\"package_edge_ngram\",\"search_analyzer\":\"package\"},\"raw\":{\"type\":\"text\",\"analyzer\":\"raw\"}}},\"version\":{\"type\":\"text\",\"index\":false},\"date\":{\"type\":\"date\",\"index\":false},\"links\":{\"type\":\"object\",\"properties\":{\"npm\":{\"type\":\"text\",\"index\":false},\"homepage\":{\"type\":\"text\",\"index\":false},\"repository\":{\"type\":\"text\",\"index\":false},\"bugs\":{\"type\":\"text\",\"index\":false}}},\"author\":{\"type\":\"object\",\"properties\":{\"name\":{\"type\":\"keyword\",\"normalizer\":\"raw\"},\"username\":{\"type\":\"keyword\",\"normalizer\":\"raw\"},\"email\":{\"type\":\"keyword\",\"normalizer\":\"raw\"},\"url\":{\"type\":\"text\",\"index\":false}}},\"publisher\":{\"type\":\"object\",\"properties\":{\"username\":{\"type\":\"text\",\"index\":false},\"email\":{\"type\":\"text\",\"index\":false}}},\"maintainers\":{\"type\":\"object\",\"properties\":{\"username\":{\"type\":\"keyword\",\"normalizer\":\"raw\"},\"email\":{\"type\":\"keyword\",\"normalizer\":\"raw\"}}},\"scope\":{\"type\":\"keyword\",\"normalizer\":\"raw\"}}},\"flags\":{\"type\":\"object\",\"properties\":{\"deprecated\":{\"type\":\"text\",\"index_options\":\"docs\"},\"insecure\":{\"type\":\"integer\"},\"unstable\":{\"type\":\"boolean\"}}},\"evaluation\":{\"type\":\"object\",\"properties\":{\"quality\":{\"type\":\"object\",\"properties\":{\"carefulness\":{\"type\":\"double\",\"index\":false},\"tests\":{\"type\":\"double\",\"index\":false},\"health\":{\"type\":\"double\",\"index\":false},\"branding\":{\"type\":\"double\",\"index\":false}}},\"popularity\":{\"type\":\"object\",\"properties\":{\"communityInterest\":{\"type\":\"double\",\"index\":false},\"downloadsCount\":{\"type\":\"double\",\"index\":false},\"downloadsAcceleration\":{\"type\":\"double\",\"index\":false},\"dependentsCount\":{\"type\":\"double\",\"index\":false}}},\"maintenance\":{\"type\":\"object\",\"properties\":{\"releasesFrequency\":{\"type\":\"double\",\"index\":false},\"commitsFrequency\":{\"type\":\"double\",\"index\":false},\"openIssues\":{\"type\":\"double\",\"index\":false},\"issuesDistribution\":{\"type\":\"double\",\"index\":false}}}}},\"score\":{\"type\":\"object\",\"properties\":{\"final\":{\"type\":\"double\"},\"detail\":{\"type\":\"object\",\"properties\":{\"quality\":{\"type\":\"double\"},\"popularity\":{\"type\":\"double\"},\"maintenance\":{\"type\":\"double\"}}}}}}}}}","statusCode":400,"response":"{\"error\":{\"root_cause\":[{\"type\":\"mapper_parsing_exception\",\"reason\":\"Root mapping definition has unsupported parameters: [score : {dynamic=false, _all={enabled=false}, properties={evaluation={type=object, properties={popularity={type=object, properties={downloadsCount={index=false, type=double}, dependentsCount={index=false, type=double}, communityInterest={index=false, type=double}, downloadsAcceleration={index=false, type=double}}}, maintenance={type=object, properties={releasesFrequency={index=false, type=double}, commitsFrequency={index=false, type=double}, issuesDistribution={index=false, type=double}, openIssues={index=false, type=double}}}, quality={type=object, properties={tests={index=false, type=double}, branding={index=false, type=double}, health={index=false, type=double}, carefulness={index=false, type=double}}}}}, score={type=object, properties={final={type=double}, detail={type=object, properties={popularity={type=double}, maintenance={type=double}, quality={type=double}}}}}, package={type=object, properties={date={index=false, type=date}, maintainers={type=object, properties={email={normalizer=raw, type=keyword}, username={normalizer=raw, type=keyword}}}, keywords={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, raw={analyzer=raw, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, author={type=object, properties={name={normalizer=raw, type=keyword}, email={normalizer=raw, type=keyword}, url={index=false, type=text}, username={normalizer=raw, type=keyword}}}, scope={normalizer=raw, type=keyword}, name={type=text, fields={standard={analyzer=standard, type=text}, autocomplete={search_analyzer=package, analyzer=package_autocomplete, type=text}, english={analyzer=package_english, type=text}, raw={normalizer=raw, type=keyword}, english_aggressive={analyzer=package_english_aggressive, type=text}, autocomplete_highlight={search_analyzer=package, analyzer=package_autocomplete_highlight, type=text, index_options=offsets}, autocomplete_keyword={search_analyzer=package_autocomplete_keyword_search, analyzer=package_autocomplete_keyword, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, description={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, publisher={type=object, properties={email={index=false, type=text}, username={index=false, type=text}}}, links={type=object, properties={bugs={index=false, type=text}, npm={index=false, type=text}, repository={index=false, type=text}, homepage={index=false, type=text}}}, version={index=false, type=text}}}, flags={type=object, properties={unstable={type=boolean}, deprecated={type=text, index_options=docs}, insecure={type=integer}}}}}]\"}],\"type\":\"mapper_parsing_exception\",\"reason\":\"Failed to parse mapping [_doc]: Root mapping definition has unsupported parameters: [score : {dynamic=false, _all={enabled=false}, properties={evaluation={type=object, properties={popularity={type=object, properties={downloadsCount={index=false, type=double}, dependentsCount={index=false, type=double}, communityInterest={index=false, type=double}, downloadsAcceleration={index=false, type=double}}}, maintenance={type=object, properties={releasesFrequency={index=false, type=double}, commitsFrequency={index=false, type=double}, issuesDistribution={index=false, type=double}, openIssues={index=false, type=double}}}, quality={type=object, properties={tests={index=false, type=double}, branding={index=false, type=double}, health={index=false, type=double}, carefulness={index=false, type=double}}}}}, score={type=object, properties={final={type=double}, detail={type=object, properties={popularity={type=double}, maintenance={type=double}, quality={type=double}}}}}, package={type=object, properties={date={index=false, type=date}, maintainers={type=object, properties={email={normalizer=raw, type=keyword}, username={normalizer=raw, type=keyword}}}, keywords={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, raw={analyzer=raw, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, author={type=object, properties={name={normalizer=raw, type=keyword}, email={normalizer=raw, type=keyword}, url={index=false, type=text}, username={normalizer=raw, type=keyword}}}, scope={normalizer=raw, type=keyword}, name={type=text, fields={standard={analyzer=standard, type=text}, autocomplete={search_analyzer=package, analyzer=package_autocomplete, type=text}, english={analyzer=package_english, type=text}, raw={normalizer=raw, type=keyword}, english_aggressive={analyzer=package_english_aggressive, type=text}, autocomplete_highlight={search_analyzer=package, analyzer=package_autocomplete_highlight, type=text, index_options=offsets}, autocomplete_keyword={search_analyzer=package_autocomplete_keyword_search, analyzer=package_autocomplete_keyword, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, description={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, publisher={type=object, properties={email={index=false, type=text}, username={index=false, type=text}}}, links={type=object, properties={bugs={index=false, type=text}, npm={index=false, type=text}, repository={index=false, type=text}, homepage={index=false, type=text}}}, version={index=false, type=text}}}, flags={type=object, properties={unstable={type=boolean}, deprecated={type=text, index_options=docs}, insecure={type=integer}}}}}]\",\"caused_by\":{\"type\":\"mapper_parsing_exception\",\"reason\":\"Root mapping definition has unsupported parameters: [score : {dynamic=false, _all={enabled=false}, properties={evaluation={type=object, properties={popularity={type=object, properties={downloadsCount={index=false, type=double}, dependentsCount={index=false, type=double}, communityInterest={index=false, type=double}, downloadsAcceleration={index=false, type=double}}}, maintenance={type=object, properties={releasesFrequency={index=false, type=double}, commitsFrequency={index=false, type=double}, issuesDistribution={index=false, type=double}, openIssues={index=false, type=double}}}, quality={type=object, properties={tests={index=false, type=double}, branding={index=false, type=double}, health={index=false, type=double}, carefulness={index=false, type=double}}}}}, score={type=object, properties={final={type=double}, detail={type=object, properties={popularity={type=double}, maintenance={type=double}, quality={type=double}}}}}, package={type=object, properties={date={index=false, type=date}, maintainers={type=object, properties={email={normalizer=raw, type=keyword}, username={normalizer=raw, type=keyword}}}, keywords={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, raw={analyzer=raw, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, author={type=object, properties={name={normalizer=raw, type=keyword}, email={normalizer=raw, type=keyword}, url={index=false, type=text}, username={normalizer=raw, type=keyword}}}, scope={normalizer=raw, type=keyword}, name={type=text, fields={standard={analyzer=standard, type=text}, autocomplete={search_analyzer=package, analyzer=package_autocomplete, type=text}, english={analyzer=package_english, type=text}, raw={normalizer=raw, type=keyword}, english_aggressive={analyzer=package_english_aggressive, type=text}, autocomplete_highlight={search_analyzer=package, analyzer=package_autocomplete_highlight, type=text, index_options=offsets}, autocomplete_keyword={search_analyzer=package_autocomplete_keyword_search, analyzer=package_autocomplete_keyword, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, description={type=text, fields={standard={analyzer=standard, type=text}, english={analyzer=package_english, type=text}, english_aggressive={analyzer=package_english_aggressive, type=text}, edge_ngram={search_analyzer=package, analyzer=package_edge_ngram, type=text}}}, publisher={type=object, properties={email={index=false, type=text}, username={index=false, type=text}}}, links={type=object, properties={bugs={index=false, type=text}, npm={index=false, type=text}, repository={index=false, type=text}, homepage={index=false, type=text}}}, version={index=false, type=text}}}, flags={type=object, properties={unstable={type=boolean}, deprecated={type=text, index_options=docs}, insecure={type=integer}}}}}]\"}},\"status\":400}"},"msg":"Scoring cycle failed","v":1}</span><br></pre></td></tr></table></figure><p>it shows failed.</p><p>Since couchDB alwasys down during my test. I think it must be something wrong with my couchDb, so I can not get my score.</p>]]></content>
<tags>
<tag> npm </tag>
<tag> couchDB </tag>
<tag> rabbitMQ </tag>
<tag> Elasticsearch </tag>
</tags>
</entry>
<entry>
<title>npm analyzer project: part 4</title>
<link href="/2019/11/06/npm-analyzer-project-part-4/"/>
<url>/2019/11/06/npm-analyzer-project-part-4/</url>
<content type="html"><![CDATA[<h2 id="Install-elasticsearch"><a href="#Install-elasticsearch" class="headerlink" title="Install elasticsearch"></a>Install elasticsearch</h2><p>elasticsearch is a scalable and speedy search, analytics, and storage.</p><h4 id="import-signing-key-PGP-key"><a href="#import-signing-key-PGP-key" class="headerlink" title="import signing key (PGP key)"></a>import signing key (PGP key)</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -</span><br><span class="line">OK</span><br></pre></td></tr></table></figure><h4 id="install-from-apt-repo"><a href="#install-from-apt-repo" class="headerlink" title="install from apt repo"></a>install from apt repo</h4><p>install the apt-transport-https package on Debian before proceeding.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ sudo apt-get install apt-transport-https</span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">apt-transport-https is already the newest version (1.8.2).</span><br><span class="line">0 upgraded, 0 newly installed, 0 to remove and 11 not upgraded.</span><br></pre></td></tr></table></figure><p>Save the repository definition to /etc/apt/sources.list.d/elastic-7.x.list</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ echo "deb https://artifacts.elastic.co/packages/7.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-7.x.list</span><br><span class="line">deb https://artifacts.elastic.co/packages/7.x/apt stable main</span><br></pre></td></tr></table></figure><p>check source list</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:/etc/apt/sources.list.d$ ls -al</span><br><span class="line">total 24</span><br><span class="line">drwxr-xr-x 2 root root 4096 Nov 6 18:05 .</span><br><span class="line">drwxr-xr-x 7 root root 4096 Nov 6 18:02 ..</span><br><span class="line">-rw-r--r-- 1 root root 419 Nov 2 06:19 bintray.rabbitmq.list</span><br><span class="line">-rw-r--r-- 1 root root 62 Nov 6 18:05 elastic-7.x.list</span><br><span class="line">-rw-r--r-- 1 root root 189 Oct 17 04:04 google-chrome.list</span><br><span class="line">-rw-r--r-- 1 root root 55 Oct 17 04:01 google-chrome.list.save</span><br></pre></td></tr></table></figure><h4 id="install-the-Elasticsearch-Debian-package"><a href="#install-the-Elasticsearch-Debian-package" class="headerlink" title="install the Elasticsearch Debian package"></a>install the Elasticsearch Debian package</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ sudo apt-get update && sudo apt-get install elasticsearch</span><br><span class="line">Get:1 http://security.debian.org/debian-security buster/updates InRelease [39.1 kB]</span><br><span class="line">Hit:2 http://deb.debian.org/debian buster InRelease </span><br><span class="line">Get:3 https://download.docker.com/linux/debian buster InRelease [44.4 kB] </span><br><span class="line">Get:4 https://artifacts.elastic.co/packages/7.x/apt stable InRelease [7,124 B] </span><br><span class="line">Ign:5 http://dl.google.com/linux/chrome/deb stable InRelease </span><br><span class="line">Hit:6 https://deb.nodesource.com/node_10.x buster InRelease </span><br><span class="line">Hit:7 http://dl.google.com/linux/chrome/deb stable Release </span><br><span class="line">Ign:8 https://dl.bintray.com/rabbitmq-erlang/debian bionic InRelease </span><br><span class="line">Get:9 http://ftp.de.debian.org/debian buster-backports InRelease [46.7 kB]</span><br><span class="line">Ign:10 https://dl.bintray.com/rabbitmq/debian bionic InRelease</span><br><span class="line">Get:11 https://dl.bintray.com/rabbitmq-erlang/debian bionic Release [12.6 kB]</span><br><span class="line">Get:12 https://dl.bintray.com/rabbitmq/debian bionic Release [74.5 kB]</span><br><span class="line">Get:13 http://security.debian.org/debian-security buster/updates/main Sources [84.1 kB]</span><br><span class="line">Get:14 http://security.debian.org/debian-security buster/updates/main amd64 Packages [112 kB]</span><br><span class="line">Get:15 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 Packages [21.9 kB]</span><br><span class="line">Get:19 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages.diff/Index [27.8 kB] </span><br><span class="line">Get:20 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-02-1415.02.pdiff [178 B] </span><br><span class="line">Get:21 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-03-0215.42.pdiff [276 B] </span><br><span class="line">Get:22 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-03-0818.43.pdiff [15.7 kB] </span><br><span class="line">Get:23 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-03-2017.32.pdiff [501 B] </span><br><span class="line">Get:24 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-05-1414.24.pdiff [5,640 B] </span><br><span class="line">Get:25 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-06-1414.27.pdiff [237 B] </span><br><span class="line">Get:25 http://ftp.de.debian.org/debian buster-backports/main amd64 Packages 2019-11-06-1414.27.pdiff [237 B] </span><br><span class="line">Fetched 493 kB in 7s (72.2 kB/s) </span><br><span class="line">Reading package lists... Done</span><br><span class="line">Reading package lists... Done</span><br><span class="line">Building dependency tree </span><br><span class="line">Reading state information... Done</span><br><span class="line">The following NEW packages will be installed:</span><br><span class="line"> elasticsearch</span><br><span class="line">0 upgraded, 1 newly installed, 0 to remove and 11 not upgraded.</span><br><span class="line">Need to get 289 MB of archives.</span><br><span class="line">After this operation, 488 MB of additional disk space will be used.</span><br><span class="line">Get:1 https://artifacts.elastic.co/packages/7.x/apt stable/main amd64 elasticsearch amd64 7.4.2 [289 MB]</span><br><span class="line">Fetched 289 MB in 29s (10.0 MB/s) </span><br><span class="line">Selecting previously unselected package elasticsearch.</span><br><span class="line">(Reading database ... 57026 files and directories currently installed.)</span><br><span class="line">Preparing to unpack .../elasticsearch_7.4.2_amd64.deb ...</span><br><span class="line">Creating elasticsearch group... OK</span><br><span class="line">Creating elasticsearch user... OK</span><br><span class="line">Unpacking elasticsearch (7.4.2) ...</span><br><span class="line">Setting up elasticsearch (7.4.2) ...</span><br><span class="line">Created elasticsearch keystore in /etc/elasticsearch</span><br><span class="line">Processing triggers for systemd (241-7~deb10u1) ...</span><br></pre></td></tr></table></figure><a id="more"></a><h4 id="running-elasticsearch-with-systemd"><a href="#running-elasticsearch-with-systemd" class="headerlink" title="running elasticsearch with systemd"></a>running elasticsearch with systemd</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ sudo /bin/systemctl daemon-reload</span><br><span class="line">vagrant@cli:~/npm_analyzer$ sudo /bin/systemctl enable elasticsearch.service</span><br><span class="line">Synchronizing state of elasticsearch.service with SysV service script with /lib/systemd/systemd-sysv-install.</span><br><span class="line">Executing: /lib/systemd/systemd-sysv-install enable elasticsearch</span><br><span class="line">Created symlink /etc/systemd/system/multi-user.target.wants/elasticsearch.service → /lib/systemd/system/elasticsearch.service.</span><br></pre></td></tr></table></figure><p>By default the Elasticsearch service doesn’t log information in the systemd journal. To enable journalctl logging, the –quiet option must be removed from the ExecStart command line in the elasticsearch.service file.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:/lib/systemd/system$ vim elasticsearch.service</span><br><span class="line">ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid --quiet</span><br></pre></td></tr></table></figure><p>delete the “–quiet”</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">ExecStart=/usr/share/elasticsearch/bin/elasticsearch -p ${PID_DIR}/elasticsearch.pid</span><br></pre></td></tr></table></figure><p>Note: do not forget to deamon-reload and restart elasticserch.service</p><h4 id="check-the-journal"><a href="#check-the-journal" class="headerlink" title="check the journal"></a>check the journal</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:/lib/systemd/system$ sudo journalctl --unit elasticsearch</span><br><span class="line">-- Logs begin at Tue 2019-11-05 07:29:08 GMT, end at Wed 2019-11-06 18:25:17 GMT. --</span><br><span class="line">-- No entries --</span><br></pre></td></tr></table></figure><h4 id="check-elasticsearch-is-running"><a href="#check-elasticsearch-is-running" class="headerlink" title="check elasticsearch is running"></a>check elasticsearch is running</h4><p>You can test that your Elasticsearch node is running by sending an HTTP request to port 9200 on localhost:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">root@cli:/home/vagrant/npm_analyzer# curl localhost:9200</span><br><span class="line">{</span><br><span class="line"> "name" : "cli",</span><br><span class="line"> "cluster_name" : "elasticsearch",</span><br><span class="line"> "cluster_uuid" : "S9dLNZ_rTpa30PX5UT3VJg",</span><br><span class="line"> "version" : {</span><br><span class="line"> "number" : "7.4.2",</span><br><span class="line"> "build_flavor" : "default",</span><br><span class="line"> "build_type" : "deb",</span><br><span class="line"> "build_hash" : "2f90bbf7b93631e52bafb59b3b049cb44ec25e96",</span><br><span class="line"> "build_date" : "2019-10-28T20:40:44.881551Z",</span><br><span class="line"> "build_snapshot" : false,</span><br><span class="line"> "lucene_version" : "8.2.0",</span><br><span class="line"> "minimum_wire_compatibility_version" : "6.8.0",</span><br><span class="line"> "minimum_index_compatibility_version" : "6.0.0-beta1"</span><br><span class="line"> },</span><br><span class="line"> "tagline" : "You Know, for Search"</span><br><span class="line">}</span><br></pre></td></tr></table></figure><h2 id="Add-these-configurations-to-the-elasticsearch-yml"><a href="#Add-these-configurations-to-the-elasticsearch-yml" class="headerlink" title="Add these configurations to the elasticsearch.yml"></a>Add these configurations to the elasticsearch.yml</h2><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">root@cli:/etc/elasticsearch# echo "action.auto_create_index: -npms-current,-npms-new,+*" >> elasticsearch.yml</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br></pre></td><td class="code"><pre><span class="line">root@cli:/etc/elasticsearch# ls</span><br><span class="line">elasticsearch.keystoreelasticsearch.yml jvm.optionslog4j2.properties role_mapping.yml roles.ymlusers users_roles</span><br><span class="line">root@cli:/etc/elasticsearch# cat elasticsearch.yml </span><br><span class="line"># ======================== Elasticsearch Configuration =========================</span><br><span class="line">#</span><br><span class="line"># NOTE: Elasticsearch comes with reasonable defaults for most settings.</span><br><span class="line"># Before you set out to tweak and tune the configuration, make sure you</span><br><span class="line"># understand what are you trying to accomplish and the consequences.</span><br><span class="line">#</span><br><span class="line"># The primary way of configuring a node is via this file. This template lists</span><br><span class="line"># the most important settings you may want to configure for a production cluster.</span><br><span class="line">#</span><br><span class="line"># Please consult the documentation for further information on configuration options:</span><br><span class="line"># https://www.elastic.co/guide/en/elasticsearch/reference/index.html</span><br><span class="line">#</span><br><span class="line"># ---------------------------------- Cluster -----------------------------------</span><br><span class="line">#</span><br><span class="line"># Use a descriptive name for your cluster:</span><br><span class="line">#</span><br><span class="line">#cluster.name: my-application</span><br><span class="line">#</span><br><span class="line"># ------------------------------------ Node ------------------------------------</span><br><span class="line">#</span><br><span class="line"># Use a descriptive name for the node:</span><br><span class="line">#</span><br><span class="line">#node.name: node-1</span><br><span class="line">#</span><br><span class="line"># Add custom attributes to the node:</span><br><span class="line">#</span><br><span class="line">#node.attr.rack: r1</span><br><span class="line">#</span><br><span class="line"># ----------------------------------- Paths ------------------------------------</span><br><span class="line">#</span><br><span class="line"># Path to directory where to store the data (separate multiple locations by comma):</span><br><span class="line">#</span><br><span class="line">path.data: /var/lib/elasticsearch</span><br><span class="line">#</span><br><span class="line"># Path to log files:</span><br><span class="line">#</span><br><span class="line">path.logs: /var/log/elasticsearch</span><br><span class="line">#</span><br><span class="line"># ----------------------------------- Memory -----------------------------------</span><br><span class="line">#</span><br><span class="line"># Lock the memory on startup:</span><br><span class="line">#</span><br><span class="line">#bootstrap.memory_lock: true</span><br><span class="line">#</span><br><span class="line"># Make sure that the heap size is set to about half the memory available</span><br><span class="line"># on the system and that the owner of the process is allowed to use this</span><br><span class="line"># limit.</span><br><span class="line">#</span><br><span class="line"># Elasticsearch performs poorly when the system is swapping the memory.</span><br><span class="line">#</span><br><span class="line"># ---------------------------------- Network -----------------------------------</span><br><span class="line">#</span><br><span class="line"># Set the bind address to a specific IP (IPv4 or IPv6):</span><br><span class="line">#</span><br><span class="line">#network.host: 192.168.0.1</span><br><span class="line">#</span><br><span class="line"># Set a custom port for HTTP:</span><br><span class="line">#</span><br><span class="line">#http.port: 9200</span><br><span class="line">#</span><br><span class="line"># For more information, consult the network module documentation.</span><br><span class="line">#</span><br><span class="line"># --------------------------------- Discovery ----------------------------------</span><br><span class="line">#</span><br><span class="line"># Pass an initial list of hosts to perform discovery when this node is started:</span><br><span class="line"># The default list of hosts is ["127.0.0.1", "[::1]"]</span><br><span class="line">#</span><br><span class="line">#discovery.seed_hosts: ["host1", "host2"]</span><br><span class="line">#</span><br><span class="line"># Bootstrap the cluster using an initial set of master-eligible nodes:</span><br><span class="line">#</span><br><span class="line">#cluster.initial_master_nodes: ["node-1", "node-2"]</span><br><span class="line">#</span><br><span class="line"># For more information, consult the discovery and cluster formation module documentation.</span><br><span class="line">#</span><br><span class="line"># ---------------------------------- Gateway -----------------------------------</span><br><span class="line">#</span><br><span class="line"># Block initial recovery after a full cluster restart until N nodes are started:</span><br><span class="line">#</span><br><span class="line">#gateway.recover_after_nodes: 3</span><br><span class="line">#</span><br><span class="line"># For more information, consult the gateway module documentation.</span><br><span class="line">#</span><br><span class="line"># ---------------------------------- Various -----------------------------------</span><br><span class="line">#</span><br><span class="line"># Require explicit names when deleting indices:</span><br><span class="line">#</span><br><span class="line">#action.destructive_requires_name: true</span><br><span class="line">action.auto_create_index: -npms-current,-npms-new,+*</span><br></pre></td></tr></table></figure><h2 id="Install-the-head-to-perform-various-manual-operations-in-a-web-GUI"><a href="#Install-the-head-to-perform-various-manual-operations-in-a-web-GUI" class="headerlink" title="Install the head to perform various manual operations in a web GUI"></a>Install the head to perform various manual operations in a web GUI</h2><p>elasticsearch-head is a web front end for an Elasticsearch cluster</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">root@cli:/home/vagrant/npm_analyzer# git clone git://github.com/mobz/elasticsearch-head.git</span><br><span class="line">Cloning into 'elasticsearch-head'...</span><br><span class="line">remote: Enumerating objects: 77, done.</span><br><span class="line">remote: Counting objects: 100% (77/77), done.</span><br><span class="line">remote: Compressing objects: 100% (57/57), done.</span><br><span class="line">remote: Total 4337 (delta 38), reused 45 (delta 17), pack-reused 4260</span><br><span class="line">Receiving objects: 100% (4337/4337), 2.51 MiB | 3.80 MiB/s, done.</span><br><span class="line">Resolving deltas: 100% (2411/2411), done.</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br></pre></td><td class="code"><pre><span class="line">root@cli:/home/vagrant/npm_analyzer# cd elasticsearch-head/</span><br><span class="line">root@cli:/home/vagrant/npm_analyzer/elasticsearch-head# npm install</span><br><span class="line">npm WARN deprecated [email protected]: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)</span><br><span class="line">npm WARN deprecated [email protected]: Use the built-in module in node 9.0.0 or newer, instead</span><br><span class="line">npm WARN deprecated [email protected]: this package is now deprecated</span><br><span class="line">npm WARN deprecated [email protected]: Please use the native JSON object instead of JSON 3</span><br><span class="line">npm WARN deprecated [email protected]: Please use the native JSON object instead of JSON 3</span><br><span class="line"></span><br><span class="line">> [email protected] install /home/vagrant/npm_analyzer/elasticsearch-head/node_modules/phantomjs-prebuilt</span><br><span class="line">> node install.js</span><br><span class="line"></span><br><span class="line">PhantomJS not found on PATH</span><br><span class="line">Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2</span><br><span class="line">Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2</span><br><span class="line">Receiving...</span><br><span class="line"> [======================================--] 96%</span><br><span class="line">Received 22866K total.</span><br><span class="line">Extracting tar contents (via spawned process)</span><br><span class="line">Removing /home/vagrant/npm_analyzer/elasticsearch-head/node_modules/phantomjs-prebuilt/lib/phantom</span><br><span class="line">Copying extracted folder /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1573068758515/phantomjs-2.1.1-linux-x86_64 -> /home/vagrant/npm_analyzer/elasticsearch-head/node_modules/phantomjs-prebuilt/lib/phantom</span><br><span class="line">Writing location.js file</span><br><span class="line">Done. Phantomjs binary available at /home/vagrant/npm_analyzer/elasticsearch-head/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs</span><br><span class="line"></span><br><span class="line">> [email protected] postinstall /home/vagrant/npm_analyzer/elasticsearch-head/node_modules/core-js</span><br><span class="line">> node postinstall || echo "ignore"</span><br><span class="line"></span><br><span class="line">Thank you for using core-js ( https://github.com/zloirock/core-js ) for polyfilling JavaScript standard library!</span><br><span class="line"></span><br><span class="line">The project needs your help! Please consider supporting of core-js on Open Collective or Patreon: </span><br><span class="line">> https://opencollective.com/core-js </span><br><span class="line">> https://www.patreon.com/zloirock </span><br><span class="line"></span><br><span class="line">Also, the author of core-js ( https://github.com/zloirock ) is looking for a good job -)</span><br><span class="line"></span><br><span class="line">npm notice created a lockfile as package-lock.json. You should commit this file.</span><br><span class="line">npm WARN [email protected] license should be a valid SPDX license expression</span><br><span class="line">npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):</span><br><span class="line">npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})</span><br><span class="line"></span><br><span class="line">added 512 packages from 620 contributors and audited 1768 packages in 115.069s</span><br><span class="line">found 40 vulnerabilities (19 low, 2 moderate, 19 high)</span><br><span class="line"> run `npm audit fix` to fix them, or `npm audit` for details</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> ╭────────────────────────────────────────────────────────────────╮</span><br><span class="line"> │ │</span><br><span class="line"> │ New minor version of npm available! 6.9.0 → 6.12.1 │</span><br><span class="line"> │ Changelog: https://github.com/npm/cli/releases/tag/v6.12.1 │</span><br><span class="line"> │ Run npm install -g npm to update! │</span><br><span class="line"> │ │</span><br><span class="line"> ╰────────────────────────────────────────────────────────────────╯</span><br></pre></td></tr></table></figure><h4 id="start-a-local-webserver-running-on-port-9100-serving-elasticsearch-head"><a href="#start-a-local-webserver-running-on-port-9100-serving-elasticsearch-head" class="headerlink" title="start a local webserver running on port 9100 serving elasticsearch-head"></a>start a local webserver running on port 9100 serving elasticsearch-head</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line">root@cli:/home/vagrant/npm_analyzer/elasticsearch-head# curl localhost:9100</span><br><span class="line"><!DOCTYPE html></span><br><span class="line"></span><br><span class="line"><html></span><br><span class="line"><head></span><br><span class="line"><meta charset="UTF-8"></span><br><span class="line"><title>elasticsearch-head</title></span><br><span class="line"><link rel="stylesheet" href="_site/base/reset.css"></span><br><span class="line"><link rel="stylesheet" href="_site/vendor.css"></span><br><span class="line"><link rel="stylesheet" href="_site/app.css"></span><br><span class="line"><script src="_site/i18n.js" data-baseDir="_site/lang" data-langs="en,fr,pt,zh,zh-TW,tr,ja"></script></span><br><span class="line"><script src="_site/vendor.js"></script></span><br><span class="line"><script src="_site/app.js"></script></span><br><span class="line"><script></span><br><span class="line">window.onload = function() {</span><br><span class="line">if(location.href.contains("/_plugin/")) {</span><br><span class="line">var base_uri = location.href.replace(/_plugin\/.*/, '');</span><br><span class="line">}</span><br><span class="line">var args = location.search.substring(1).split("&").reduce(function(r, p) {</span><br><span class="line">r[decodeURIComponent(p.split("=")[0])] = decodeURIComponent(p.split("=")[1]); return r;</span><br><span class="line">}, {});</span><br><span class="line">new app.App("body", {</span><br><span class="line">id: "es",</span><br><span class="line">base_uri: args["base_uri"] || base_uri,</span><br><span class="line">auth_user : args["auth_user"] || "",</span><br><span class="line">auth_password : args["auth_password"],</span><br><span class="line">dashboard: args["dashboard"]</span><br><span class="line">});</span><br><span class="line">};</span><br><span class="line"></script></span><br><span class="line"><link rel="icon" href="_site/base/favicon.png" type="image/png"></span><br><span class="line"></head></span><br><span class="line"><body></body></span><br><span class="line"></html></span><br></pre></td></tr></table></figure><p>Note: In order to enable CORS in elasticsearch, so I add these commands to /etc/elasticsearch/elasticsearch.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">http.cors.enabled: true</span><br><span class="line">http.cors.allow-origin: "*"</span><br></pre></td></tr></table></figure><p>also need to modify the network host in elasticsearch.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">network.host: 0.0.0.0</span><br><span class="line">discovery.seed_hosts: ["0.0.0.0", "127.0.0.1"]</span><br></pre></td></tr></table></figure><p>then restart the elasticsearch</p><p>go to localhost:9100, you can connect to elasticsearch</p><img src="/2019/11/06/npm-analyzer-project-part-4/elastic.png" class="">]]></content>
<tags>
<tag> npm </tag>
<tag> Elasticsearch </tag>
</tags>
</entry>
<entry>
<title>npm analyzer project: part 3</title>
<link href="/2019/11/05/npm-analyzer-project-part-3/"/>
<url>/2019/11/05/npm-analyzer-project-part-3/</url>
<content type="html"><![CDATA[<h2 id="RabbitMQ-is-the-most-widely-deployed-open-source-message-broker"><a href="#RabbitMQ-is-the-most-widely-deployed-open-source-message-broker" class="headerlink" title="RabbitMQ is the most widely deployed open source message broker."></a><a href="https://www.rabbitmq.com/" target="_blank" rel="noopener">RabbitMQ</a> is the most widely deployed open source message broker.</h2><p>At first I go to <a href="https://www.rabbitmq.com/" target="_blank" rel="noopener">RabbitMQ</a> to download and install it.<br>I use the shell script to install rabbitMQ</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line">#!/bin/sh</span><br><span class="line"></span><br><span class="line">## If sudo is not available on the system,</span><br><span class="line">## uncomment the line below to install it</span><br><span class="line"># apt-get install -y sudo</span><br><span class="line"></span><br><span class="line">sudo apt-get update -y</span><br><span class="line"></span><br><span class="line">## Install prerequisites</span><br><span class="line">sudo apt-get install curl gnupg -y</span><br><span class="line"></span><br><span class="line">## Install RabbitMQ signing key</span><br><span class="line">curl -fsSL https://github.com/rabbitmq/signing-keys/releases/download/2.0/rabbitmq-release-signing-key.asc | sudo apt-key add -</span><br><span class="line"></span><br><span class="line">## Install apt HTTPS transport</span><br><span class="line">sudo apt-get install apt-transport-https</span><br><span class="line"></span><br><span class="line">## Add Bintray repositories that provision latest RabbitMQ and Erlang 21.x releases</span><br><span class="line">sudo tee /etc/apt/sources.list.d/bintray.rabbitmq.list <<EOF</span><br><span class="line">## Installs the latest Erlang 21.x release.</span><br><span class="line">## Change component to "erlang" to install the latest version (22.x or later).</span><br><span class="line">## "bionic" as distribution name should work for any later Ubuntu or Debian release.</span><br><span class="line">## See the release to distribution mapping table in RabbitMQ doc guides to learn more.</span><br><span class="line">deb https://dl.bintray.com/rabbitmq-erlang/debian bionic erlang-21.x</span><br><span class="line">deb https://dl.bintray.com/rabbitmq/debian bionic main</span><br><span class="line">EOF</span><br><span class="line"></span><br><span class="line">## Update package indices</span><br><span class="line">sudo apt-get update -y</span><br><span class="line"></span><br><span class="line">## Install rabbitmq-server and its dependencies</span><br><span class="line">sudo apt-get install rabbitmq-server -y --fix-missing</span><br></pre></td></tr></table></figure><p>after run this instll shell script, it doesn’t work. I try docker image.</p><h4 id="Install-rabbitMQ-docker-image"><a href="#Install-rabbitMQ-docker-image" class="headerlink" title="Install rabbitMQ docker image"></a>Install rabbitMQ docker image</h4><p>according to <a href="https://hub.docker.com/_/rabbitmq" target="_blank" rel="noopener">rabbitMQ</a>, run command</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker run -d --hostname my-rabbit --name some-rabbit -p 8080:15672 rabbitmq:3-management</span><br></pre></td></tr></table></figure><a id="more"></a><img src="/2019/11/05/npm-analyzer-project-part-3/dockerrun.png" class=""><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker logs some-rabbit</span><br></pre></td></tr></table></figure><p>then I saw the logs like this:</p><img src="/2019/11/05/npm-analyzer-project-part-3/logs.png" class=""><h4 id="login-docker-rabbitmq-container-to-see-config-file"><a href="#login-docker-rabbitmq-container-to-see-config-file" class="headerlink" title="login docker rabbitmq container to see config file"></a>login docker rabbitmq container to see config file</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker exec -it some-rabbit bash</span><br></pre></td></tr></table></figure><img src="/2019/11/05/npm-analyzer-project-part-3/config.png" class=""><h4 id="login-localhost-15672-on-browser"><a href="#login-localhost-15672-on-browser" class="headerlink" title="login localhost:15672 on browser"></a>login localhost:15672 on browser</h4><img src="/2019/11/05/npm-analyzer-project-part-3/15672.png" class=""><p>using guest/guest log in</p><img src="/2019/11/05/npm-analyzer-project-part-3/login.png" class="">]]></content>
<tags>
<tag> npm </tag>
<tag> rabbitMQ </tag>
</tags>
</entry>
<entry>
<title>npm analyzer project: part 2</title>
<link href="/2019/11/05/npm-analyzer-project-part-2/"/>
<url>/2019/11/05/npm-analyzer-project-part-2/</url>
<content type="html"><![CDATA[<h2 id="CouchDB"><a href="#CouchDB" class="headerlink" title="CouchDB"></a>CouchDB</h2><p><a href="https://couchdb.apache.org/" target="_blank" rel="noopener">Apache CouchDB</a> lets you access your data where you need it. The Couch Replication Protocol is implemented in a variety of projects and products that span every imaginable computing environment from globally distributed server-clusters, over mobile phones to web browsers.<br>Since I am working in treehouses/cli vagrant in which CouchDB docker already installed, I need to run couchDB docker.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">docker run -d -p 5984:984 --name=vmnet8 treehouses/couchdb:2.3.1</span><br></pre></td></tr></table></figure><p>then open localhost:5984 on my browser</p><img src="/2019/11/05/npm-analyzer-project-part-2/couch5984.png" class=""><p>It shows couchDB installed</p><h4 id="Create-a-new-database-named-npms"><a href="#Create-a-new-database-named-npms" class="headerlink" title="Create a new database named npms"></a>Create a new database named npms</h4><img src="/2019/11/05/npm-analyzer-project-part-2/database_npms.jpg" class=""><p>visit localhost:5984/_utils, you can see the new database npms is created</p><img src="/2019/11/05/npm-analyzer-project-part-2/npms.jpg" class=""><h4 id="Setup-npm-replication-from-https-replicate-npmjs-com-registry-to-npm-database-in-continuous-mode"><a href="#Setup-npm-replication-from-https-replicate-npmjs-com-registry-to-npm-database-in-continuous-mode" class="headerlink" title="Setup npm replication from https://replicate.npmjs.com/registry to npm database in continuous mode"></a>Setup npm replication from <a href="https://replicate.npmjs.com/registry" target="_blank" rel="noopener">https://replicate.npmjs.com/registry</a> to npm database in continuous mode</h4><img src="/2019/11/05/npm-analyzer-project-part-2/replication.jpg" class=""><a id="more"></a><p>waiting for a while, the replication will be done</p><img src="/2019/11/05/npm-analyzer-project-part-2/replication_done.jpg" class=""><h4 id="For-the-replication-to-work-I-might-need-to-add-some-CouchDB-config"><a href="#For-the-replication-to-work-I-might-need-to-add-some-CouchDB-config" class="headerlink" title="For the replication to work, I might need to add some CouchDB config:"></a>For the replication to work, I might need to add some CouchDB config:</h4><img src="/2019/11/05/npm-analyzer-project-part-2/create_plugin.jpg" class=""><p>you can see in the replication config file, the auth plugin added</p><img src="/2019/11/05/npm-analyzer-project-part-2/add_plugin.jpg" class=""><h4 id="Creating-the-document-design-npms-analyzer-in-the-npms-database-with-the-contents-of-https-github-com-npms-io-npms-analyzer-blob-master-config-couchdb-npms-analyzer-json"><a href="#Creating-the-document-design-npms-analyzer-in-the-npms-database-with-the-contents-of-https-github-com-npms-io-npms-analyzer-blob-master-config-couchdb-npms-analyzer-json" class="headerlink" title="Creating the document _design/npms-analyzer in the npms database with the contents of https://github.com/npms-io/npms-analyzer/blob/master/config/couchdb/npms-analyzer.json"></a>Creating the document _design/npms-analyzer in the npms database with the contents of <a href="https://github.com/npms-io/npms-analyzer/blob/master/config/couchdb/npms-analyzer.json" target="_blank" rel="noopener">https://github.com/npms-io/npms-analyzer/blob/master/config/couchdb/npms-analyzer.json</a></h4><img src="/2019/11/05/npm-analyzer-project-part-2/document.png" class="">]]></content>
<tags>
<tag> npm </tag>
<tag> couchDB </tag>
<tag> rabbitMQ </tag>
<tag> Elasticsearch </tag>
</tags>
</entry>
<entry>
<title>npms analyzer project: part 1</title>
<link href="/2019/11/02/npms-analyzer-project-part-1/"/>
<url>/2019/11/02/npms-analyzer-project-part-1/</url>
<content type="html"><![CDATA[<p>The npms-analyzer analyzes the npm ecosystem, collecting info, evaluating and scoring each package.In this project, I am going to create an experimental enviroment for treehouses cli package.</p><h2 id="Set-up-all-the-items-needed-for-this-project"><a href="#Set-up-all-the-items-needed-for-this-project" class="headerlink" title="Set up all the items needed for this project."></a>Set up all the items needed for this project.</h2><h3 id="Config-file"><a href="#Config-file" class="headerlink" title="Config file"></a>Config file</h3><p>Configure Node.js Applications</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">npm i config</span><br></pre></td></tr></table></figure><h3 id="Programs-amp-utilities"><a href="#Programs-amp-utilities" class="headerlink" title="Programs & utilities"></a>Programs & utilities</h3><h4 id="node"><a href="#node" class="headerlink" title="node"></a>node</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ which node</span><br><span class="line">/usr/bin/node</span><br></pre></td></tr></table></figure><h4 id="git"><a href="#git" class="headerlink" title="git"></a>git</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ which git</span><br><span class="line">/usr/bin/git</span><br></pre></td></tr></table></figure><h4 id="rm-mkdir-chmod-wc"><a href="#rm-mkdir-chmod-wc" class="headerlink" title="rm mkdir chmod wc"></a>rm mkdir chmod wc</h4><p>They are already in Ubuntu.</p><h4 id="tar"><a href="#tar" class="headerlink" title="tar"></a>tar</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vagrant@cli:~/npm_analyzer$ which tar</span><br><span class="line">/usr/bin/tar</span><br></pre></td></tr></table></figure><h4 id="pino"><a href="#pino" class="headerlink" title="pino"></a>pino</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">npm install -g pino-prteey</span><br><span class="line">vagrant@cli:~/npm_analyzer$ which pino-pretty</span><br><span class="line">/usr/bin/pino-pretty</span><br><span class="line">vagrant@cli:~/npm_analyzer$ ls -l /usr/bin/pino-pretty</span><br><span class="line">lrwxrwxrwx 1 root root 38 Nov 1 17:25 /usr/bin/pino-pretty -> ../lib/node_modules/pino-pretty/bin.js</span><br></pre></td></tr></table></figure><p>to be contiuned</p>]]></content>
<tags>
<tag> npm </tag>
<tag> couchDB </tag>
<tag> rabbitMQ </tag>
<tag> Elasticsearch </tag>
</tags>
</entry>
<entry>
<title>Create npm repo</title>
<link href="/2019/10/19/Create-npm-repo/"/>
<url>/2019/10/19/Create-npm-repo/</url>
<content type="html"><![CDATA[<p>I need to simulate a npm repo to test codes. The following steps are how to create a npm repo.</p><h4 id="Create-a-npm-account"><a href="#Create-a-npm-account" class="headerlink" title="Create a npm account"></a>Create a npm account</h4><p>Go to <a href="npmjs.com">npmjs.com</a>, click “join for free”, then create your account. The weird thing is the password needs at least 10 characters!</p><h4 id="Create-a-npm-repo-on-your-local-terminal"><a href="#Create-a-npm-repo-on-your-local-terminal" class="headerlink" title="Create a npm repo on your local terminal"></a>Create a npm repo on your local terminal</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">mkdir npm_own</span><br></pre></td></tr></table></figure><h4 id="Copy-github-com-treehouses-cli-package-json-url-to-local"><a href="#Copy-github-com-treehouses-cli-package-json-url-to-local" class="headerlink" title="Copy github.com/treehouses/cli package.json url to local"></a>Copy github.com/treehouses/cli package.json url to local</h4><p>Because I want to test the code on github.com/treehouses/cli and don’t want to mess up the current repo, I copy the url to my local</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wget https://github.com/treehouses/cli/blob/master/package.json</span><br></pre></td></tr></table></figure><h4 id="Create-your-own-package-json"><a href="#Create-your-own-package-json" class="headerlink" title="Create your own package.json"></a>Create your own package.json</h4><p>I try to create a repo named “flyingsaucer8”.<br>Using vim to make my own package.json based on treehouses package.json</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line">{</span><br><span class="line"> "name": "flyingsaucer8",</span><br><span class="line"> "version": "0.0.6",</span><br><span class="line"> "description": "flyingsaucer",</span><br><span class="line"> "main": "cli.sh",</span><br><span class="line"> "bin": {</span><br><span class="line"> "flyingsaucer": "cli.sh"</span><br><span class="line"> },</span><br><span class="line"> "publishConfig": {</span><br><span class="line"> "access": "public"</span><br><span class="line"> },</span><br><span class="line"> "repository": {</span><br><span class="line"> "type": "git",</span><br><span class="line"> "url": "https://github.com/vmnet8/flyingsaucer.git"</span><br><span class="line"> },</span><br><span class="line"> "scripts": {</span><br><span class="line"> "postinstall": "if [ $(id -u) = 0 ]; then ln -sr _flyingsaucer /etc/bash_completion.d/_flyingsaucer; fi && exit 0",</span><br><span class="line"> "postuninstall": "if [ $(id -u) = 0 ]; then rm /etc/bash_completion.d/_flyingsaucer; fi && exit 0",</span><br><span class="line"> "test": "echo \"Error: no test specified\" && exit 0"</span><br><span class="line"> },</span><br><span class="line"> "keywords": [</span><br><span class="line"> "flyingsaucer"</span><br><span class="line"> ],</span><br><span class="line"> "author": {</span><br><span class="line"> "name": "flyingsaucer team",</span><br><span class="line"> "email": "[email protected]",</span><br><span class="line"> "url": "https://flyingsaucer.io"</span><br><span class="line"> },</span><br><span class="line"> "license": "AGPL-3.0",</span><br><span class="line"> "bugs": {</span><br><span class="line"> "url": "https://github.com/vmnet8/flyingsaucer/issues",</span><br><span class="line"> "email": "[email protected]"</span><br><span class="line"> },</span><br><span class="line"> "homepage": "https://github.com/vmnet8/flyingsaucer",</span><br><span class="line"> "dependencies": {}</span><br><span class="line">}</span><br></pre></td></tr></table></figure><a id="more"></a><h4 id="npm-publish"><a href="#npm-publish" class="headerlink" title="npm publish"></a>npm publish</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">pi@treehouses:~/git_repo/npm_own:npm login</span><br><span class="line">Username:vmnet8</span><br><span class="line">password:</span><br><span class="line">Email:(This IS Public)[email protected]</span><br><span class="line">Logged in as vmnet8 on https://registry.npmjs.org/.</span><br></pre></td></tr></table></figure><p>then you can publish your repo to npm</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pi@treehouses:~/git_repo/npm_own:npm publish --access=public</span><br></pre></td></tr></table></figure><p>Right now, you know how to push your code to npmjs repo</p><h3 id="copy-other-code-from-github-com-treehouses-cli"><a href="#copy-other-code-from-github-com-treehouses-cli" class="headerlink" title="copy other code from github.com/treehouses/cli"></a>copy other code from github.com/treehouses/cli</h3><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">wget https://github.com/treehouses/cli/blob/master/cli.sh</span><br><span class="line">wget https://github.com/treehouses/cli/blob/master/_treehouses</span><br></pre></td></tr></table></figure><p>I need to copy globals.sh and other modules, I only copied speedtest and detectpi.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">wget https://github.com/treehouses/cli/blob/master/modules/speedtest.sh</span><br><span class="line">wget https://github.com/treehouses/cli/blob/master/modules/globals.sh</span><br><span class="line">wget https:eehouses/cli/blob/master/modules/detectrpi.sh</span><br></pre></td></tr></table></figure><h4 id="change-the-cli-sh-and-treehouses-to-fit-my-situation"><a href="#change-the-cli-sh-and-treehouses-to-fit-my-situation" class="headerlink" title="change the cli.sh and _treehouses to fit my situation"></a>change the cli.sh and _treehouses to fit my situation</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br></pre></td><td class="code"><pre><span class="line">#!/bin/bash</span><br><span class="line"></span><br><span class="line">SCRIPTPATH=$(realpath "$0")</span><br><span class="line">SCRIPTFOLDER=$(dirname "$SCRIPTPATH")</span><br><span class="line"></span><br><span class="line">source "$SCRIPTFOLDER/modules/globals.sh" </span><br><span class="line">source "$SCRIPTFOLDER/modules/detectrpi.sh" </span><br><span class="line">source "$SCRIPTFOLDER/modules/speedtest.sh"</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line">case $1 in</span><br><span class="line"> detectrpi)</span><br><span class="line"> detectrpi</span><br><span class="line"> ;;</span><br><span class="line"> speedtest)</span><br><span class="line"> shift</span><br><span class="line"> speedtest "$@"</span><br><span class="line"> ;;</span><br><span class="line"> help)</span><br><span class="line"> help "$2"</span><br><span class="line"> ;;</span><br><span class="line"> *)</span><br><span class="line"> help</span><br><span class="line"> ;;</span><br><span class="line">esac</span><br></pre></td></tr></table></figure><p>Notice: I am supposed to keep globals.sh</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br></pre></td><td class="code"><pre><span class="line">#!/bin/bash</span><br><span class="line"></span><br><span class="line">_treehouses_complete()</span><br><span class="line">{</span><br><span class="line"> local cur prev</span><br><span class="line"></span><br><span class="line"> # Valid top-level completions</span><br><span class="line"> commands="detectpi speedtest"</span><br><span class="line"> help_cmds="detectpi speedtest"</span><br><span class="line"></span><br><span class="line"># services_cmds="" Think about it, a little bit complicated</span><br><span class="line"></span><br><span class="line"># upgrade_cmds="" I am not sure whether we should put autocompletion here.</span><br><span class="line"></span><br><span class="line"></span><br><span class="line"></span><br><span class="line"> COMPREPLY=()</span><br><span class="line"> cur=${COMP_WORDS[COMP_CWORD]}</span><br><span class="line"> prev=${COMP_WORDS[COMP_CWORD-1]}</span><br><span class="line"></span><br><span class="line"> if [ $COMP_CWORD -eq 1 ]</span><br><span class="line"> then</span><br><span class="line"> COMPREPLY=( $(compgen -W "${commands}" -- $cur) )</span><br><span class="line"> elif [ $COMP_CWORD -eq 2 ]</span><br><span class="line"> then</span><br><span class="line"> case "$prev" in</span><br><span class="line"> "help")</span><br><span class="line"> COMPREPLY=( $(compgen -W "$help_cmds" -- $cur) )</span><br><span class="line"> ;;</span><br><span class="line"> "*")</span><br><span class="line"> ;;</span><br><span class="line"> esac</span><br><span class="line"> elif [ $COMP_CWORD -eq 3 ]</span><br><span class="line"> then</span><br><span class="line"> case "$prev" in</span><br><span class="line"> "console")</span><br><span class="line"> COMPREPLY=( $(compgen -W "$bootoption_second_cmds" -- $cur) )</span><br><span class="line"> ;;</span><br><span class="line"> "desktop")</span><br><span class="line"> COMPREPLY=( $(compgen -W "$bootoption_second_cmds" -- $cur) )</span><br><span class="line"> ;;</span><br><span class="line"> "notice")</span><br><span class="line"> COMPREPLY=( $(compgen -W "$notice_cmds" -- $cur) )</span><br><span class="line"> ;;</span><br><span class="line"> "*")</span><br><span class="line"> ;;</span><br><span class="line"> esac</span><br><span class="line"> fi</span><br><span class="line"></span><br><span class="line">}</span><br><span class="line">complete -F _treehouses_complete treehouses</span><br></pre></td></tr></table></figure><p>then I need to add execute to cli.sh and _flyingsaucer</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">pi@treehosues:~/git_repo/npm_own/ $chmod +x cli.sh</span><br><span class="line">pi@treehosues:~/git_repo/npm_own/ $chmod +x _flyingsaucer</span><br></pre></td></tr></table></figure><h4 id="install-npm-package"><a href="#install-npm-package" class="headerlink" title="install npm package"></a>install npm package</h4><p>Go to my local terminal to install npm package</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pi@treehosues:~/git_repo/npm_own/ $ sudo npm install -g flyingsaucer8</span><br></pre></td></tr></table></figure><h4 id="test"><a href="#test" class="headerlink" title="test"></a>test</h4><p>After npm installed, I can test on my local maching if the bash completion works</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"></span><br><span class="line">pi@treehouses:~/git_repo/npm_own/ $ cd /etc/bash_completion</span><br><span class="line">pi@treehouses:~/etc/bash_completion $ ls -al</span><br><span class="line">lrwxrwxrwx 1 root root 54 Oct 17 07:09 _treehouses -> ../../usr/lib/node_modules/@treehouses/cli/_treehouses</span><br></pre></td></tr></table></figure><p>It shows the symbolic doesn’t work.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"> </span><br><span class="line">pi@treehouses:~/git_repo/npm_own/ $ flyingsaucer speedtest</span><br><span class="line">pi@treehouses:~/git_repo/npm_own/ $ flyingsaucer detectpi</span><br></pre></td></tr></table></figure><p>It shows <code>speedtest</code> works well, but <code>detectpi</code> doesn’t</p>]]></content>
<tags>
<tag> npm </tag>
</tags>
</entry>
<entry>
<title>Convert markdown file to PDF file</title>
<link href="/2019/10/14/Convert-markdown-file-to-PDF-file/"/>
<url>/2019/10/14/Convert-markdown-file-to-PDF-file/</url>
<content type="html"><![CDATA[<p>I keep my resume on github, and is going to update it with my working experience changing.Since resume is written in markdown format on github, I need a PDF format when applying job on line. How to convert markdown file to PDF file easily?</p><h2 id="Convert-markdown-file-to-html-format-first"><a href="#Convert-markdown-file-to-html-format-first" class="headerlink" title="Convert markdown file to html format first"></a>Convert markdown file to html format first</h2><p>You need to check markdown is already installed or not. If not, you need use <code>apt install markdown</code> to install it.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~$ which markdown</span><br><span class="line">/usr/bin/markdown</span><br></pre></td></tr></table></figure><p>Go to directory where your markdown file lies, run</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~/git_repo/resume$ markdown resume.md resume.html</span><br></pre></td></tr></table></figure><p>you will convert resume.md markdown file to resume.html</p><p>then you can go to webbrowser to see this html file, to make sure it works.</p><h2 id="Convert-html-file-to-PDF-file"><a href="#Convert-html-file-to-PDF-file" class="headerlink" title="Convert html file to PDF file"></a>Convert html file to PDF file</h2><p>To convert PDF file, you need to use wkhtmltopdf tool</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">apt install wkhtmltopdf</span><br><span class="line">anna@ubuntu1804:~$ which wkhtmltopdf</span><br><span class="line">/usr/bin/wkhtmltopdf</span><br></pre></td></tr></table></figure><p>go to the same directory where the resume.md and resume.html lie, run</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">wkhtmltopdf -s letter -B 25mm -T 25mm -L 25mm -R 25mm resume.html resume.pdf</span><br></pre></td></tr></table></figure><p>then you can get the PDF format resume file.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~/git_repo/resume$ ll</span><br><span class="line">-rw-r--r-- 1 anna anna 4493 Oct 14 22:46 resume.html</span><br><span class="line">-rw-r--r-- 1 anna anna 3728 Oct 14 22:45 resume.md</span><br><span class="line">-rw-r--r-- 1 anna anna 34985 Oct 14 22:46 resume.pdf</span><br></pre></td></tr></table></figure><h2 id="Write-a-script-to-execute-convert-automaticly"><a href="#Write-a-script-to-execute-convert-automaticly" class="headerlink" title="Write a script to execute convert automaticly"></a>Write a script to execute convert automaticly</h2><a id="more"></a><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">#/bin/bash</span><br><span class="line"></span><br><span class="line">md=$1</span><br><span class="line"></span><br><span class="line"># validate md</span><br><span class="line">if [ ! -f "$md" ]; then</span><br><span class="line"> echo "Usage: $0 <markdown file name>"</span><br><span class="line"> exit 1</span><br><span class="line">fi</span><br><span class="line"></span><br><span class="line">html="${md%%.*}.html"</span><br><span class="line"></span><br><span class="line">pdf="${md%%.*}.pdf"</span><br><span class="line"></span><br><span class="line">markdown $md > $html</span><br><span class="line"></span><br><span class="line">wkhtmltopdf -s letter -B 25mm -T 25mm -L 25mm -R 25mm $html $pdf</span><br></pre></td></tr></table></figure><p>after you update resume, you can run md2pdf.sh, then get the PDF file automaticlly</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">anna@ubuntu1804:~/git_repo/resume$ ./md2pdf.sh resume.md</span><br><span class="line">Loading page (1/2)</span><br><span class="line">Printing pages (2/2) </span><br><span class="line">Done </span><br><span class="line">anna@ubuntu1804:~/git_repo/resume$ ll</span><br><span class="line">total 76</span><br><span class="line">drwxr-xr-x 3 anna anna 4096 Oct 15 00:17 ./</span><br><span class="line">drwxr-xr-x 3 anna anna 4096 Oct 14 21:38 ../</span><br><span class="line">drwxr-xr-x 8 anna anna 4096 Oct 15 00:09 .git/</span><br><span class="line">-rw-r--r-- 1 anna anna 12 Oct 14 22:48 .gitignore</span><br><span class="line">-rwxr-xr-x 1 anna anna 268 Oct 14 22:36 md2pdf.sh*</span><br><span class="line">-rw-r--r-- 1 anna anna 17 Oct 14 21:38 README.md</span><br><span class="line">-rw-r--r-- 1 anna anna 4774 Oct 15 00:18 resume.html</span><br><span class="line">-rw-r--r-- 1 anna anna 3974 Oct 15 00:17 resume.md</span><br><span class="line">-rw-r--r-- 1 anna anna 36063 Oct 15 00:18 resume.pdf</span><br><span class="line">-rw-r--r-- 1 anna anna 280 Oct 14 21:38 support.md</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> shell script </tag>
</tags>
</entry>
<entry>
<title>Ovirt-configuring-storage</title>
<link href="/2019/10/04/Ovirt-configuring-storage/"/>
<url>/2019/10/04/Ovirt-configuring-storage/</url>
<content type="html"><![CDATA[<p>Last time we talked about adding a host to oVirt engine, it looks like following:</p><img src="/2019/10/04/Ovirt-configuring-storage/first.jpg" class=""><h2 id="configure-storage"><a href="#configure-storage" class="headerlink" title="configure storage"></a>configure storage</h2><p>Setting up storage is a prerequisite for a new data center because a data center cannot be initialized unless storage domains are attached and activated.<br>A storage domain is a collection of images that have a common storage domain contains complete images of templates and virtual machots), ISO files, and metadata about themselves. A storage domain can be made of either block devices (SAN - iSCSI or FCP) or a file system (NAS - NFS, GlusterFS, or other POSIX compliant file systems).</p><h2 id="configure-NFS-storage"><a href="#configure-NFS-storage" class="headerlink" title="configure NFS storage"></a>configure NFS storage</h2><p>First I use another centOS as a nfs server,then attached it to storage domain.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@cube4200 ~]# showmount -e</span><br><span class="line">Export list for cube4200:</span><br><span class="line">/export/hosted_vm 192.168.0.0/24</span><br></pre></td></tr></table></figure><img src="/2019/10/04/Ovirt-configuring-storage/create_nfs.jpg" class=""><p>It is attached successfully.</p><a id="more"></a><img src="/2019/10/04/Ovirt-configuring-storage/finish_nfs.jpg" class=""><h2 id="configure-iSCSI-storage"><a href="#configure-iSCSI-storage" class="headerlink" title="configure iSCSI storage"></a>configure iSCSI storage</h2><p>First I am going to use targetcli to create an iSCSI on my centOS.</p><p>after that, I go back to engine to attach it, but it said it can’t find the lun.I tried several times but failed.<br>After searching online, I found there is a specical user named vdsm whose user id is 36, group idis 36. This is a special user who supports engine to mount iSCSI to storage domain.Without this user the iSCSI can’t be found.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[root@cube4200 ~]# id vdsm</span><br><span class="line">uid=36(vdsm) gid=36(kvm) groups=36(kvm),179(sanlock),107(qemu)</span><br></pre></td></tr></table></figure><p>after adding user vdsm on iSCSI server, engine can find the lun.</p><img src="/2019/10/04/Ovirt-configuring-storage/create_lun.jpg" class=""><img src="/2019/10/04/Ovirt-configuring-storage/all_done.jpg" class="">]]></content>
<tags>
<tag> Ovirt </tag>
</tags>
</entry>
<entry>
<title>Ovirt: installation -- using cockpit wizard</title>
<link href="/2019/09/14/Ovirt-installation/"/>
<url>/2019/09/14/Ovirt-installation/</url>
<content type="html"><![CDATA[<h2 id="Ovirt"><a href="#Ovirt" class="headerlink" title="Ovirt"></a>Ovirt</h2><p>oVirt is an open-source distributed virtualization solution, designed to manage your entire enterprise infrastructure. oVirt uses the trusted KVM hypervisor and is built upon several other community projects, including libvirt, Gluster, PatternFly, and Ansible.<br>Go to <a href="https://www.ovirt.org/download/" target="_blank" rel="noopener">Ovirt</a>to see how to install Ovirt.<br>oVirt Engine and a Host are installed together with the Engine running as a Virtual Machine on that Host.I use cockpit wizard to install Ovirt on CentOS.<br>Two types hosts</p><ul><li>oVirt node, other name ‘thin host’, it is a minimal OS, contains the only packagesrequired for the machine to act as a host. </li><li>centOS or RHEL host<br>I use centOS 7 as a host.<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum update -y</span><br></pre></td></tr></table></figure>Add the official oVirt repository:<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm</span><br></pre></td></tr></table></figure>I use cockpit wizard to install Ovirt on CentOS.<br>Install Cockpit and the cockpit-ovirt-dashboard plugin:<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum install cockpit cockpit-ovirt-dashboard -y</span><br></pre></td></tr></table></figure>Enable Cockpit:<figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">systemctl enable --now cockpit.socket</span><br></pre></td></tr></table></figure>Open the firewall:<a id="more"></a><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">firewall-cmd --add-service=cockpit</span><br><span class="line">firewall-cmd --add-service=cockpit --permanent</span><br></pre></td></tr></table></figure>Access the Administration Portal using a web browser,log in with user name ‘admin’ and password at https://[Host IP or FQDN]:9090.<br>click Virtualization → Hosted Engine to add a host</li></ul><p>To be continued…</p>]]></content>
<tags>
<tag> Ovirt </tag>
</tags>
</entry>
<entry>
<title>git and github</title>
<link href="/2019/09/09/git-and-github/"/>
<url>/2019/09/09/git-and-github/</url>
<content type="html"><![CDATA[<h4 id="Git-is-a-software-and-be-installed-on-different-OS-such-as-Linux-Mac-Windows-Github-is-a-web-server-which-provide-a-tool-for-people-to-use-on-a-website"><a href="#Git-is-a-software-and-be-installed-on-different-OS-such-as-Linux-Mac-Windows-Github-is-a-web-server-which-provide-a-tool-for-people-to-use-on-a-website" class="headerlink" title="Git is a software and be installed on different OS, such as Linux, Mac, Windows. Github is a web server which provide a tool for people to use on a website."></a>Git is a software and be installed on different OS, such as Linux, Mac, Windows. Github is a web server which provide a tool for people to use on a website.</h4><h5 id="on-Github"><a href="#on-Github" class="headerlink" title="on Github"></a>on Github</h5><p>Create a repository<br>Create a new file<br>Commit (same as save)<br>Issue (like comments)<br>Create an organization<br>Create branch<br>Pull request<br>Merge</p><h4 id="Most-of-the-time-you-need-to-work-on-your-local-machine-to-use-git-You-can-clone-your-repo-from-github"><a href="#Most-of-the-time-you-need-to-work-on-your-local-machine-to-use-git-You-can-clone-your-repo-from-github" class="headerlink" title="Most of the time, you need to work on your local machine to use git. You can clone your repo from github."></a>Most of the time, you need to work on your local machine to use git. You can clone your repo from github.</h4><h5 id="On-local-computer"><a href="#On-local-computer" class="headerlink" title="On local computer"></a>On local computer</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">git clone [email protected]:vmnet8/Hello-World.git</span><br><span class="line">Cloning into 'Hello-World'...</span><br><span class="line">remote: Enumerating objects: 4, done.</span><br><span class="line">remote: Counting objects: 100% (4/4), done.</span><br><span class="line">remote: Compressing objects: 100% (3/3), done.</span><br><span class="line">remote: Total 14 (delta 0), reused 0 (delta 0), pack-reused 10</span><br><span class="line">Receiving objects: 100% (14/14), done.</span><br><span class="line">Resolving deltas: 100% (1/1), done.</span><br><span class="line">Checking connectivity... done.</span><br></pre></td></tr></table></figure><p>make this a repo located on your local OS<br>compare with github repo you clone</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">git status</span><br><span class="line">On branch master</span><br><span class="line">Your branch is up-to-date with 'origin/master'.</span><br><span class="line">nothing to commit, working directory clean</span><br></pre></td></tr></table></figure><h5 id="After-you-make-some-changes-on-your-local-computer-you-need-to-save-it-into-repo-it-is-not-just-save-it-needs-two-steps-to-“save”-into-repo-for-example-add-a-new-line-in-“README”"><a href="#After-you-make-some-changes-on-your-local-computer-you-need-to-save-it-into-repo-it-is-not-just-save-it-needs-two-steps-to-“save”-into-repo-for-example-add-a-new-line-in-“README”" class="headerlink" title="After you make some changes on your local computer, you need to save it into repo, it is not just save, it needs two steps to “save” into repo.for example, add a new line in “README”"></a>After you make some changes on your local computer, you need to save it into repo, it is not just save, it needs two steps to “save” into repo.for example, add a new line in “README”</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">git status</span><br><span class="line">On branch master</span><br><span class="line">Your branch is up-to-date with 'origin/master'.</span><br><span class="line">Changes not staged for commit:</span><br><span class="line"> (use "git add <file>..." to update what will be committed)</span><br><span class="line"> (use "git checkout -- <file>..." to discard changes in working directory)</span><br><span class="line"></span><br><span class="line"> modified: README.md</span><br><span class="line"></span><br><span class="line">no changes added to commit (use "git add" and/or "git commit -a")</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">git add README.md</span><br></pre></td></tr></table></figure><p>put this changed file to a stage to wait to be saved in the future.<br>any changes if want to be saved should be added to this stage first!</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">git commit README.md -m "add new line under bottom"</span><br><span class="line">[master 37c5b60] add new line under bottom</span><br><span class="line"> 1 file changed, 1 insertion(+)</span><br></pre></td></tr></table></figure><p>only done this, the changed can be “saved” into repo</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">git remote -v</span><br><span class="line">origin [email protected]:vmnet8/Hello-World.git (fetch)</span><br><span class="line">origin [email protected]:vmnet8/Hello-World.git (push)</span><br></pre></td></tr></table></figure><p>Then you can make the changes you made on your local computer to sync to github repo</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">git push origin master</span><br><span class="line">Counting objects: 3, done.</span><br><span class="line">Delta compression using up to 2 threads.</span><br><span class="line">Compressing objects: 100% (2/2), done.</span><br><span class="line">Writing objects: 100% (3/3), 302 bytes | 0 bytes/s, done.</span><br><span class="line">Total 3 (delta 1), reused 0 (delta 0)</span><br><span class="line">remote: Resolving deltas: 100% (1/1), completed with 1 local object.</span><br><span class="line">To [email protected]:vmnet8/Hello-World.git</span><br><span class="line"> efdcef9..37c5b60 master -> master</span><br></pre></td></tr></table></figure><p>If you made changes on github first, how to get your local to sync to github repo?</p><a id="more"></a><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">git pull origin master </span><br><span class="line">remote: Enumerating objects: 4, done.</span><br><span class="line">remote: Counting objects: 100% (4/4), done.</span><br><span class="line">remote: Compressing objects: 100% (3/3), done.</span><br><span class="line">remote: Total 3 (delta 0), reused 0 (delta 0), pack-reused 0</span><br><span class="line">Unpacking objects: 100% (3/3), done.</span><br><span class="line">From github.com:vmnet8/Hello-World</span><br><span class="line"> * branch master -> FETCH_HEAD</span><br><span class="line"> 37c5b60..c99e3a4 master -> origin/master</span><br><span class="line">Updating 37c5b60..c99e3a4</span><br><span class="line">Fast-forward</span><br><span class="line"> new file from github | 1 +</span><br><span class="line"> 1 file changed, 1 insertion(+)</span><br><span class="line"> create mode 100644 new file from github</span><br></pre></td></tr></table></figure><h4 id="Using-other-person’s-repo"><a href="#Using-other-person’s-repo" class="headerlink" title="Using other person’s repo"></a>Using other person’s repo</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">git clone [email protected]:RainbowCoder/Rainbow-Song.git</span><br><span class="line">Cloning into 'Rainbow-Song'...</span><br><span class="line">remote: Enumerating objects: 9, done.</span><br><span class="line">remote: Total 9 (delta 0), reused 0 (delta 0), pack-reused 9</span><br><span class="line">Receiving objects: 100% (9/9), done.</span><br><span class="line">Resolving deltas: 100% (1/1), done.</span><br><span class="line">Checking connectivity... done.</span><br></pre></td></tr></table></figure><p>you can just make changes on your local, but can’t push it to others repo.<br>You can sync from other’s repo, but just one direction, from others to your local, not from your change to others.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">git pull origin master </span><br><span class="line">From github.com:RainbowCoder/Rainbow-Song</span><br><span class="line"> * branch master -> FETCH_HEAD</span><br><span class="line">Already up-to-date.</span><br></pre></td></tr></table></figure><p>If you first fork others’s repo to your github, then clone your github url to your local, you can make changes and pull or push to your github. On your github you can pull request to the original repo.</p><h4 id="about-branch"><a href="#about-branch" class="headerlink" title="about branch"></a>about branch</h4><p>show the present branch you are</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">git branch </span><br><span class="line"> branch_newfile</span><br><span class="line">* master</span><br></pre></td></tr></table></figure><p>change to other branch</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">git checkout branch_newfile</span><br><span class="line">Switched to branch 'branch_newfile'</span><br><span class="line">Your branch is up-to-date with 'origin/branch_newfile'.</span><br></pre></td></tr></table></figure><p>If you want to see the commit history on local</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">git log</span><br><span class="line">commit a43ab8dd9725911488d4e33f4518e9c03c8ee538</span><br><span class="line">Merge: 12021a5 99e5eb0</span><br><span class="line">Author: Anna X <[email protected]></span><br><span class="line">Date: Sat Sep 7 17:06:46 2019 -0700</span><br><span class="line"></span><br><span class="line"> all</span><br><span class="line"></span><br><span class="line">commit 12021a54f4ab8be1dad823cf428e9b57cac98e0b</span><br><span class="line">Merge: 9dce990 eb4ac1f</span><br><span class="line">Author: Anna X <[email protected]></span><br><span class="line">Date: Sat Sep 7 16:41:52 2019 -0700</span><br><span class="line"></span><br><span class="line"> commit all changes</span><br></pre></td></tr></table></figure><h5 id="Make-a-repo-on-your-local-computer"><a href="#Make-a-repo-on-your-local-computer" class="headerlink" title="Make a repo on your local computer"></a>Make a repo on your local computer</h5><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">mkdir new_repo</span><br><span class="line">git init</span><br><span class="line">Initialized empty Git repository in /home/anna/github_repo/new_repo/.git/</span><br><span class="line">ll</span><br><span class="line">total 12</span><br><span class="line">drwxrwxr-x 3 anna anna 4096 Sep 9 12:34 .</span><br><span class="line">drwxrwxr-x 16 anna anna 4096 Sep 9 12:34 ..</span><br><span class="line">drwxrwxr-x 7 anna anna 4096 Sep 9 12:34 .git</span><br></pre></td></tr></table></figure><p>create a .git file under this dir, it means it created a repo<br>If you want to sync to github, just login your github account, create an empty repo. Then bond the local and github repo together.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">git remote add origin [email protected]:vmnet8/new_repo.git</span><br><span class="line">git remote -v</span><br><span class="line">origin [email protected]:vmnet8/new_repo.git (fetch)</span><br><span class="line">origin [email protected]:vmnet8/new_repo.git (push)</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> git </tag>
</tags>
</entry>
<entry>
<title>Cyber Security: Some pen tests in Kali-Linux in Virtualbox</title>
<link href="/2019/04/10/Cyber-Security-Some-pen-tests-in-Kali-Linux-in-Virtualbox/"/>
<url>/2019/04/10/Cyber-Security-Some-pen-tests-in-Kali-Linux-in-Virtualbox/</url>
<content type="html"><![CDATA[<h4 id="Abstract"><a href="#Abstract" class="headerlink" title="Abstract:"></a>Abstract:</h4><p>Install Virtualbox in computer system, in which to install Kali Linux, then try wireshark, nmap and crunch application. It shows a lot of security tools can be implemented correctly in Virtualbox.</p><h4 id="Methods"><a href="#Methods" class="headerlink" title="Methods:"></a>Methods:</h4><ol><li><p>Go to <a href="www.virtualbox.org">virtualbox page</a> to download the virtual 5.2.18 version, then install it based on wizard.</p></li><li><p>Search “kali OVA”, then go to official kali linux download page, select the “Kali Linux 64 bit Vbox”, click “offensive security download page” icon to select “Kali Linux 64 bit Ova” which is for Virtualbox, download it.</p></li><li><p>Open Virtual box,click “file” button then select “import application”, input the “kali 2018.3 ova” into the frame, then select the fold where the “kali linux 64 bit” you have download before, then open it.</p></li><li><p>Select the “OS Redhat 64 bit”, choose all other items as defaults, such as sound card, network adapter card etc, then click “import”. Waiting a couple minutes to finish installing. After installation, you can see “kali 2018.3” icon on the Virtualbox panel.</p></li><li><p>Double click the “kali 2018.3”, it will start to boot, just taking defaults. Input the username “root” and password “toor” to log in Kali.</p><a id="more"></a></li><li><p>Click the top panel of Kali, select firefox browser to input “google” to create some packets, then click No.9 sniffing and spoofing to select wireshark to open it. Ignore the Lua message,then click the “eth0”.</p></li><li><p>Using Kali terminal to input command “ping 192.168.0.1” to connect router.</p></li><li><p>Check the wireshark, stop it and capture the packets.</p></li><li><p>Input “ifconfig eth0” command in Kali terminal, get the information of ip address 10.2.0.15, then go to wireshark to capture the packets. Input the “ICMP” into wireshark frame to observe just ICMP packets. then save it and get a screenshot.</p></li><li><p>In Kali machine to click nmap application. Then in terminal command line to input “nmap -v -A scanme.nmap.org” command to do OS detection and version detection, script detection of “scanme.nmap.org”, meanwhile in wireshark to start a new packet capture. using filter to limit the packets both to and from 45.33.32.156 in and out this virtual machine. Get a screenshot from wireshark.</p></li><li><p>In Kali click the “crunch” application, which can create a wordlist based on your criteria, then you can use this wordlist to a file or a program.</p></li></ol><p>In terminal line, input “crunch 1 2 > /tmp/wordlist” to test this application.</p><h4 id="Results"><a href="#Results" class="headerlink" title="Results:"></a>Results:</h4><img src="/2019/04/10/Cyber-Security-Some-pen-tests-in-Kali-Linux-in-Virtualbox/pic1.jpg" class=""><p>“Ping 192.168.0.1” command can get the response from router. It shows packets can successfully go through from virtual machine to out router. After paused the wireshark, narrow down to ICMP, wireshark shows there are totally 98 bytes, displayed 98 bytes, no dropped between 192.168.0.1 and 10.0.2.15.It shows the packets to and from inside and outside.</p><img src="/2019/04/10/Cyber-Security-Some-pen-tests-in-Kali-Linux-in-Virtualbox/pic2.jpg" class=""><p>After filter the packets just between the 10.0.2.15 and 45.33.32.156 (scanme.nmap.org) it shows 99.6% displayed which are 2828 packets in the wireshark capture screenshot.It also shows using TCP protocol to visit the <a href="www.scanme.nmap.org">www.scanme.nmap.ogr</a> successfully.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">crunch 1 2 > /tmp/wordlisttest</span><br></pre></td></tr></table></figure><p>It generated a wordlist starts from “a”, ends to “zz”, then save it as a file named “wordlisttest” in tmp folder in Kali linux.</p><img src="/2019/04/10/Cyber-Security-Some-pen-tests-in-Kali-Linux-in-Virtualbox/pic3.jpg" class=""><p>Virtualbox is a powerful application on which a lot of machine can run. In Kali, Nmap has command line interface, which is easy to use compared to GUI interface.</p>]]></content>
<tags>
<tag> cyber security </tag>
</tags>
</entry>
<entry>
<title>Linux: key points of raid</title>
<link href="/2019/03/02/Linux-key-points-of-raid/"/>
<url>/2019/03/02/Linux-key-points-of-raid/</url>
<content type="html"><![CDATA[<p>RAID stands for “Redundant Array of Independent Disks”.</p><h4 id="The-RAID-levels"><a href="#The-RAID-levels" class="headerlink" title="The RAID levels"></a>The RAID levels</h4><p>Raid 0: stripe mode,no redundancy in this level,not safe for storage.<br>Raid 1: mirror mode,having redundancy and good at backup.<br>Raid 1+0:it is the combination of RAID-1 and RAID-0.<br>Raid 5: at least 3 disks, can stand max 1 disk broken<br>Raid 6:at least 4 disks, can stand max 2 disks broken</p><h4 id="config-file"><a href="#config-file" class="headerlink" title="config file"></a>config file</h4><p>/etc/mdadm.conf<br>/proc/mdstat </p><h4 id="Create"><a href="#Create" class="headerlink" title="Create"></a>Create</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# mdadm --create /dev/md0 --auto=yes --level=5 --raid-devices=3 --chunk=64 /dev/sdb{2,3,4}</span><br><span class="line">mdadm: Defaulting to version 1.2 metadata</span><br><span class="line">mdadm: array /dev/md0 started.</span><br><span class="line">[root@client ~]# mkfs.xfs /dev/md0</span><br><span class="line">meta-data=/dev/md0 isize=512 agcount=8, agsize=65456 blks</span><br><span class="line"> = sectsz=512 attr=2, projid32bit=1</span><br><span class="line"> = crc=1 finobt=0, sparse=0</span><br><span class="line">data = bsize=4096 blocks=523648, imaxpct=25</span><br><span class="line"> = sunit=16 swidth=32 blks</span><br><span class="line">naming =version 2 bsize=4096 ascii-ci=0 ftype=1</span><br><span class="line">log =internal log bsize=4096 blocks=2560, version=2</span><br><span class="line"> = sectsz=512 sunit=16 blks, lazy-count=1</span><br><span class="line">realtime =none extsz=4096 blocks=0, rtextents=0</span><br><span class="line">[root@client mnt]# mkdir raid_md0</span><br><span class="line">[root@client mnt]# mount /dev/md0 /mnt/raid_md0/</span><br><span class="line">[root@client mnt]# df -h /dev/md0</span><br><span class="line">Filesystem Size Used Avail Use% Mounted on</span><br><span class="line">/dev/md0 2.0G 33M 2.0G 2% /mnt/raid_md0</span><br></pre></td></tr></table></figure><a id="more"></a><h4 id="check-the-status"><a href="#check-the-status" class="headerlink" title="check the status"></a>check the status</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# cat /proc/mdstat </span><br><span class="line">Personalities : [raid6] [raid5] [raid4] </span><br><span class="line">md0 : active raid5 sdb4[3] sdb3[1] sdb2[0]</span><br><span class="line"> 2095104 blocks super 1.2 level 5, 64k chunk, algorithm 2 [3/3] [UUU]</span><br><span class="line"> </span><br><span class="line">unused devices: <none></span><br><span class="line">[root@client ~]# mdadm --detail /dev/md0</span><br><span class="line">/dev/md0:</span><br><span class="line"> Version : 1.2</span><br><span class="line"> Raid Level : raid5</span><br><span class="line"> Array Size : 2095104 (2046.00 MiB 2145.39 MB)</span><br><span class="line"> Used Dev Size : 1047552 (1023.00 MiB 1072.69 MB)</span><br><span class="line"> Raid Devices : 3</span><br><span class="line"> Total Devices : 3</span><br><span class="line"> Persistence : Superblock is persistent</span><br><span class="line"></span><br><span class="line"> State : clean </span><br><span class="line"> Active Devices : 3</span><br><span class="line">Working Devices : 3</span><br><span class="line"> Failed Devices : 0</span><br><span class="line"> Spare Devices : 0</span><br><span class="line"></span><br><span class="line"> Layout : left-symmetric</span><br><span class="line"> Chunk Size : 64K</span><br><span class="line"></span><br><span class="line"> Name : client.centos7.study:0 (local to host client.centos7.study)</span><br><span class="line"> UUID : add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line"> Events : 18</span><br><span class="line"></span><br><span class="line"> Number Major Minor RaidDevice State</span><br><span class="line"> 0 8 18 0 active sync /dev/sdb2</span><br><span class="line"> 1 8 19 1 active sync /dev/sdb3</span><br><span class="line"> 3 8 20 2 active sync /dev/sdb4</span><br></pre></td></tr></table></figure><p>If mount md0 permanently, need to know md0 UUID</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@client mnt]# mdadm --detail /dev/md0 |grep -i uuid</span><br><span class="line"> UUID : add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">[root@client mnt]# vim /etc/mdadm.conf</span><br><span class="line">ARRAY /dev/md0 UUID : add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">[root@client mnt]# blkid /dev/md0</span><br><span class="line">/dev/md0: UUID="acf71d8f-8027-49fc-b3dd-fe6d89f3491f" TYPE="xfs"</span><br><span class="line">[root@client mnt]# vim /etc/fstab</span><br><span class="line">UUID="acf71d8f-8027-49fc-b3dd-fe6d89f3491f" /mnt/raid_md0 xfs defaults 0 0</span><br><span class="line">[root@client mnt]# mount -a</span><br></pre></td></tr></table></figure><h4 id="Close-raid"><a href="#Close-raid" class="headerlink" title="Close raid"></a>Close raid</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br></pre></td><td class="code"><pre><span class="line">[root@client mnt]# umount /dev/md0</span><br><span class="line">[root@client ~]# mdadm --manage /dev/md0 --fail /dev/sdb2</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: set /dev/sdb2 faulty in /dev/md0</span><br><span class="line">[root@client ~]# mdadm --manage /dev/md0 --fail /dev/sdb3</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: set /dev/sdb3 faulty in /dev/md0</span><br><span class="line">[root@client ~]# mdadm --manage /dev/md0 --fail /dev/sdb4</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: set /dev/sdb4 faulty in /dev/md0</span><br><span class="line">[root@client ~]# mdadm --detail /dev/md0</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">/dev/md0:</span><br><span class="line"> Version : 1.2</span><br><span class="line"> Creation Time : Wed Jul 17 11:37:14 2019</span><br><span class="line"> Raid Level : raid5</span><br><span class="line"> Array Size : 2095104 (2046.00 MiB 2145.39 MB)</span><br><span class="line"> Used Dev Size : 1047552 (1023.00 MiB 1072.69 MB)</span><br><span class="line"> Raid Devices : 3</span><br><span class="line"> Total Devices : 3</span><br><span class="line"> Persistence : Superblock is persistent</span><br><span class="line"></span><br><span class="line"> Update Time : Wed Jul 17 12:53:53 2019</span><br><span class="line"> State : clean, FAILED </span><br><span class="line"> Active Devices : 0</span><br><span class="line"> Failed Devices : 3</span><br><span class="line"> Spare Devices : 0</span><br><span class="line"></span><br><span class="line"> Layout : left-symmetric</span><br><span class="line"> Chunk Size : 64K</span><br><span class="line"></span><br><span class="line"> Number Major Minor RaidDevice State</span><br><span class="line"> - 0 0 0 removed</span><br><span class="line"> - 0 0 1 removed</span><br><span class="line"> - 0 0 2 removed</span><br><span class="line"></span><br><span class="line"> 0 8 18 - faulty /dev/sdb2</span><br><span class="line"> 1 8 19 - faulty /dev/sdb3</span><br><span class="line"> 3 8 20 - faulty /dev/sdb4</span><br><span class="line">[root@client ~]# mdadm --manage /dev/md0 --remove /dev/sdb2</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: hot removed /dev/sdb2 from /dev/md0</span><br><span class="line">[root@client ~]# mdadm --manage /dev/md0 --remove /dev/sdb3</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: hot removed /dev/sdb3 from /dev/md0</span><br><span class="line">[root@client ~]# mdadm --manage /dev/md0 --remove /dev/sdb4</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: hot removed /dev/sdb4 from /dev/md0</span><br><span class="line">[root@client ~]# mdadm --detail /dev/md0</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">/dev/md0:</span><br><span class="line"> Version : 1.2</span><br><span class="line"> Creation Time : Wed Jul 17 11:37:14 2019</span><br><span class="line"> Raid Level : raid5</span><br><span class="line"> Array Size : 2095104 (2046.00 MiB 2145.39 MB)</span><br><span class="line"> Used Dev Size : 1047552 (1023.00 MiB 1072.69 MB)</span><br><span class="line"> Raid Devices : 3</span><br><span class="line"> Total Devices : 0</span><br><span class="line"> Persistence : Superblock is persistent</span><br><span class="line"></span><br><span class="line"> Update Time : Wed Jul 17 12:55:39 2019</span><br><span class="line"> State : clean, FAILED </span><br><span class="line"> Active Devices : 0</span><br><span class="line"> Failed Devices : 0</span><br><span class="line"> Spare Devices : 0</span><br><span class="line"></span><br><span class="line"> Layout : left-symmetric</span><br><span class="line"> Chunk Size : 64K</span><br><span class="line"></span><br><span class="line"> Number Major Minor RaidDevice State</span><br><span class="line"> - 0 0 0 removed</span><br><span class="line"> - 0 0 1 removed</span><br><span class="line"> - 0 0 2 removed</span><br><span class="line">[root@client ~]# mdadm --stop /dev/md0</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: stopped /dev/md0</span><br><span class="line">[root@client ~]# mdadm --detail /dev/md0</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and UUID</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and :</span><br><span class="line">mdadm: only give one device per ARRAY line: /dev/md0 and add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">mdadm: ARRAY line /dev/md0 has no identity information.</span><br><span class="line">mdadm: cannot open /dev/md0: No such file or directory</span><br><span class="line">[root@client ~]# cat /proc/mdstat </span><br><span class="line">Personalities : [raid6] [raid5] [raid4] </span><br><span class="line">unused devices: <none></span><br><span class="line">[root@client ~]# lsblk</span><br><span class="line">NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT</span><br><span class="line">sdb 8:16 0 29.9G 0 disk </span><br><span class="line">├─sdb1 8:17 0 2G 0 part </span><br><span class="line">│ ├─thin_vg-thinpool_tmeta 253:4 0 4M 0 lvm </span><br><span class="line">│ │ └─thin_vg-thinpool-tpool 253:6 0 1G 0 lvm </span><br><span class="line">│ │ ├─thin_vg-thinpool 253:7 0 1G 0 lvm </span><br><span class="line">│ │ └─thin_vg-annathin1 253:8 0 10G 0 lvm </span><br><span class="line">│ └─thin_vg-thinpool_tdata 253:5 0 1G 0 lvm </span><br><span class="line">│ └─thin_vg-thinpool-tpool 253:6 0 1G 0 lvm </span><br><span class="line">│ ├─thin_vg-thinpool 253:7 0 1G 0 lvm </span><br><span class="line">│ └─thin_vg-annathin1 253:8 0 10G 0 lvm </span><br><span class="line">├─sdb2 8:18 0 1G 0 part </span><br><span class="line">├─sdb3 8:19 0 1G 0 part </span><br><span class="line">└─sdb4 8:20 0 1G 0 part </span><br><span class="line">sdc 8:32 0 40G 0 disk </span><br><span class="line">sr0 11:0 1 1024M 0 rom </span><br><span class="line">[root@client ~]# vim /etc/mdadm.conf</span><br><span class="line">#ARRAY /dev/md0 UUID : add4f325:d589a29e:1f234b1b:309885ff</span><br><span class="line">[root@client ~]# vim /etc/fstab</span><br><span class="line">#UUID="acf71d8f-8027-49fc-b3dd-fe6d89f3491f" /mnt/raid_md0 xfs defaults 0 0</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Cyber Security: Using Nmap-zenmap and Wireshark to do pen test</title>
<link href="/2019/02/27/Cyber-Security-Using-Nmap-zenmap-and-Wireshark-to-do-pen-test/"/>
<url>/2019/02/27/Cyber-Security-Using-Nmap-zenmap-and-Wireshark-to-do-pen-test/</url>
<content type="html"><![CDATA[<h4 id="Abstract"><a href="#Abstract" class="headerlink" title="Abstract:"></a>Abstract:</h4><p>Using Zenmap, enumerating 2 IP address to scan, Using Wireshark to observe the TCP handshake process when enumerating 2 IP addresses with Zenmap.</p><h4 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction:"></a>Introduction:</h4><p>Scan 2 IP with Zenmap and use Wireshark to observe the TCP handshake.</p><h4 id="Methods"><a href="#Methods" class="headerlink" title="Methods:"></a>Methods:</h4><ol><li><p>Go to <a href="www.nmap.org">nmap page</a> to download zenmap application.</p></li><li><p>Open Wireshark to see the Wi_Fi wiggling wave, it shows capturing the packets, then click it.</p></li><li><p>Open Zenmap, input first IP address 192.168.227.187 into “target” frame, then select “quick scan” to see what will happen.</p></li><li><p>go to Wireshark to stop and save it for first IP scan.</p></li><li><p>go back to Zenmap to input other IP address into “target” frame, do quick scan again.</p></li><li><p>In Wireshark, select the source and destination IP address 192.168.0.1 to observe the captured packets, analyze the TCP handshake process.</p><a id="more"></a><h4 id="Results"><a href="#Results" class="headerlink" title="Results:"></a>Results:</h4></li></ol><p>For the 192.168.227.187 address, zenmap quickly scanned 100 ports, 98 ports are closed, port 22 and port 10000 are open.</p><img src="/2019/02/27/Cyber-Security-Using-Nmap-zenmap-and-Wireshark-to-do-pen-test/ports_open.jpg" class=""><p>For the 192.168.0.1, Zenmap shows there are 4 ports open, they are ports 53,80, 8080, 8443.</p><img src="/2019/02/27/Cyber-Security-Using-Nmap-zenmap-and-Wireshark-to-do-pen-test/4_ports_open.jpg" class=""><p>In Wireshark, observing the packets from 192.168.0.1, it shows the synchronized three way handshake.</p><p>a. from the source 192.168.0.140 sent the request to 192.168.0.1, the flag is SYN<br>b. from the destination 192.168.0.1 to 192.168.0.140, it acknowledges that it receives the request and at the same time, closed the socket. The flag is RST ACK.</p><img src="/2019/02/27/Cyber-Security-Using-Nmap-zenmap-and-Wireshark-to-do-pen-test/two_way_shakehands.jpg" class=""><p>Also notice some requests from 192.168.0.140 to 192.168.0.1 didn’t get acknowledge from the destination port</p><h4 id="Discussion"><a href="#Discussion" class="headerlink" title="Discussion:"></a>Discussion:</h4><p>Sometimes there is no “three” handshakes, only “two” handshakes, because the host B is going to close the sockets after it acknowledge the packets that were sent from the host A.</p>]]></content>
<tags>
<tag> cyber security </tag>
</tags>
</entry>
<entry>
<title>Cyber Security: using wireshark to trace packets</title>
<link href="/2019/02/19/Cyber-Security-using-wireshark-to-trace-packets/"/>
<url>/2019/02/19/Cyber-Security-using-wireshark-to-trace-packets/</url>
<content type="html"><![CDATA[<p>Wireshark is a network packet analyzer. It will try to capture network packets and tries to display that packet data as detailed as possible.</p><h4 id="Abstract-Install-Wireshark-and-use-it-to-trace-the-packet-I-generated-find-my-IP-address"><a href="#Abstract-Install-Wireshark-and-use-it-to-trace-the-packet-I-generated-find-my-IP-address" class="headerlink" title="Abstract: Install Wireshark and use it to trace the packet I generated, find my IP address."></a>Abstract: Install Wireshark and use it to trace the packet I generated, find my IP address.</h4><h4 id="Methods"><a href="#Methods" class="headerlink" title="Methods:"></a>Methods:</h4><ol><li><p>Go to <a href="www.wireshark.org">wireshark pages</a>, read some basic knowledge of wireshark, then download <a href="https://1.na.dl.wireshark.org/win64/Wireshark-win64-3.0.2.exe" target="_blank" rel="noopener">wireshark Windows Installer(64-bit)</a>, according to the instructions to install it on my windows operating system.</p></li><li><p>go to desktop to open Wireshark up, double click “*Wi-Fi” link,it will show a “Capturing from my local area connection” to see the packets captured here.</p></li><li><p>Open a browser and input a website, such as <a href="http://www.qwerty.com" target="_blank" rel="noopener">www.qwerty.com</a> to generate the traffic.</p><a id="more"></a></li><li><p>Go back to wireshark to toolbar to click the red button “stop”.</p></li><li><p>Save this screen to my computer as “wireshark_1.pcapng”</p></li><li><p>go to wireshark tool bar to open “view-time display format” to choose “seconds since previous displayed packets”</p></li><li><p>go to the toolbar to open “edit-find packet” in search frame to input “qwerty”, select the “string” to narrow the search, or you can just input “dns” as search key to see what is the output is. I also click the Proctol HTTP to follow the stream.</p></li><li><p>take a screenshot</p><img src="/2019/02/19/Cyber-Security-using-wireshark-to-trace-packets/packets_captured.jpg" class=""></li><li><p>go to windows command to input “ipconfig /all “ command to get my ip address 192.168.0.140</p></li><li><p>take a screenshot</p><img src="/2019/02/19/Cyber-Security-using-wireshark-to-trace-packets/my_ip_address.jpg" class=""></li></ol><h4 id="Results"><a href="#Results" class="headerlink" title="Results:"></a>Results:</h4><p>I found my IP address in Wireshark trace<br>I also found the packets from the website I just visited and follow its HTTP stream.</p>]]></content>
<tags>
<tag> cyber security </tag>
</tags>
</entry>
<entry>
<title>Linux: LVM thin volume and snapshot</title>
<link href="/2019/02/12/Linux-LVM-thin-volume-and-snapshot/"/>
<url>/2019/02/12/Linux-LVM-thin-volume-and-snapshot/</url>
<content type="html"><![CDATA[<p>LVM: logical volume manager</p><p>Advantage: give more flexible of space</p><h4 id="LVM-thin-volume"><a href="#LVM-thin-volume" class="headerlink" title="LVM thin volume"></a>LVM thin volume</h4><p>Create thin pool first</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# lvcreate -L 1G -T thin_vg/thinpool</span><br><span class="line"> Using default stripesize 64.00 KiB.</span><br><span class="line"> Logical volume "thinpool" created.</span><br><span class="line">[root@client ~]# lvdisplay thin_vg/thinpool</span><br><span class="line"> --- Logical volume ---</span><br><span class="line"> LV Name thinpool</span><br><span class="line"> VG Name thin_vg</span><br><span class="line"> LV UUID ncc3Kh-eZgD-WLQn-F4cq-pPJJ-rYjD-XsTx34</span><br><span class="line"> LV Write Access read/write</span><br><span class="line"> LV Creation host, time client.centos7.study, 2019-07-12 13:02:42 -0700</span><br><span class="line"> LV Pool metadata thinpool_tmeta</span><br><span class="line"> LV Pool data thinpool_tdata</span><br><span class="line"> LV Status available</span><br><span class="line"> # open 0</span><br><span class="line"> LV Size 1.00 GiB</span><br><span class="line"> Allocated pool data 0.00%</span><br><span class="line"> Allocated metadata 0.98%</span><br><span class="line"> Current LE 256</span><br><span class="line"> Segments 1</span><br><span class="line"> Allocation inherit</span><br><span class="line"> Read ahead sectors auto</span><br><span class="line"> - currently set to 8192</span><br><span class="line"> Block device 253:6</span><br></pre></td></tr></table></figure><a id="more"></a><p>Create a thin1 on thinpool</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# lvcreate -V 10G -T thin_vg/thinpool -n annathin1</span><br><span class="line"> Using default stripesize 64.00 KiB.</span><br><span class="line"> WARNING: Sum of all thin volume sizes (10.00 GiB) exceeds the size of thin pool thin_vg/thinpool and the size of whole volume group (2.00 GiB)!</span><br><span class="line"> For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.</span><br><span class="line"> Logical volume "annathin1" created.</span><br><span class="line">[root@client ~]# lvdisplay thin_vg</span><br><span class="line"> --- Logical volume ---</span><br><span class="line"> LV Name thinpool</span><br><span class="line"> VG Name thin_vg</span><br><span class="line"> LV UUID ncc3Kh-eZgD-WLQn-F4cq-pPJJ-rYjD-XsTx34</span><br><span class="line"> LV Write Access read/write</span><br><span class="line"> LV Creation host, time client.centos7.study, 2019-07-12 13:02:42 -0700</span><br><span class="line"> LV Pool metadata thinpool_tmeta</span><br><span class="line"> LV Pool data thinpool_tdata</span><br><span class="line"> LV Status available</span><br><span class="line"> # open 2</span><br><span class="line"> LV Size 1.00 GiB</span><br><span class="line"> Allocated pool data 0.00%</span><br><span class="line"> Allocated metadata 1.07%</span><br><span class="line"> Current LE 256</span><br><span class="line"> Segments 1</span><br><span class="line"> Allocation inherit</span><br><span class="line"> Read ahead sectors auto</span><br><span class="line"> - currently set to 8192</span><br><span class="line"> Block device 253:6</span><br><span class="line"> </span><br><span class="line"> --- Logical volume ---</span><br><span class="line"> LV Path /dev/thin_vg/annathin1</span><br><span class="line"> LV Name annathin1</span><br><span class="line"> VG Name thin_vg</span><br><span class="line"> LV UUID q47tG0-P2xf-CMsw-a2tE-SgWl-gjni-VJD1jA</span><br><span class="line"> LV Write Access read/write</span><br><span class="line"> LV Creation host, time client.centos7.study, 2019-07-12 13:09:12 -0700</span><br><span class="line"> LV Pool name thinpool</span><br><span class="line"> LV Status available</span><br><span class="line"> # open 0</span><br><span class="line"> LV Size 10.00 GiB</span><br><span class="line"> Mapped size 0.00%</span><br><span class="line"> Current LE 2560</span><br><span class="line"> Segments 1</span><br><span class="line"> Allocation inherit</span><br><span class="line"> Read ahead sectors auto</span><br><span class="line"> - currently set to 8192</span><br><span class="line"> Block device 253:8</span><br><span class="line">[root@client /]# mkfs.xfs /dev/thin_vg/annathin1 </span><br><span class="line">meta-data=/dev/thin_vg/annathin1 isize=512 agcount=16, agsize=163824 blks</span><br><span class="line"> = sectsz=512 attr=2, projid32bit=1</span><br><span class="line"> = crc=1 finobt=0, sparse=0</span><br><span class="line">data = bsize=4096 blocks=2621184, imaxpct=25</span><br><span class="line"> = sunit=16 swidth=16 blks</span><br><span class="line">naming =version 2 bsize=4096 ascii-ci=0 ftype=1</span><br><span class="line">log =internal log bsize=4096 blocks=2560, version=2</span><br><span class="line"> = sectsz=512 sunit=16 blks, lazy-count=1</span><br><span class="line">realtime =none extsz=4096 blocks=0, rtextents=0</span><br><span class="line">[root@client /]# mkdir /mnt/annathin1</span><br><span class="line">[root@client /]# mount /dev/thin_vg/annathin1 /mnt/annathin1/</span><br><span class="line">[root@client /]# df -Th /mnt/annathin1</span><br><span class="line">Filesystem Type Size Used Avail Use% Mounted on</span><br><span class="line">/dev/mapper/thin_vg-annathin1 xfs 10G 33M 10G 1% /mnt/annathin1</span><br></pre></td></tr></table></figure><p>copy some files to annathin1</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@client ]# cp -a /etc/vsftpd/ /mnt/annathin1/</span><br><span class="line">[root@client ]# cp -a /usr/share/doc/man-db-2.6.3/ /mnt/annathin1/</span><br><span class="line">[root@client annathin1]# ll</span><br><span class="line">total 0</span><br><span class="line">drwxr-xr-x. 2 root root 113 Jun 2 2017 man-db-2.6.3</span><br><span class="line">drwxr-xr-x. 2 root root 128 Apr 4 12:46 vsftpd</span><br><span class="line">[root@client ~]# df -Th /mnt/annathin1/</span><br><span class="line">Filesystem Type Size Used Avail Use% Mounted on</span><br><span class="line">/dev/mapper/thin_vg-annathin1 xfs 10G 34M 10G 1% /mnt/annathin1</span><br></pre></td></tr></table></figure><p>The vg thin_vg actually only has 1G, but it shows 10G after create thinpool. That is the magical of LVM thinpool.<br>Note: users are not supposed to exceed actual 1G volume even though it shows having 10G volume, otherwise it will destory the filesystem!</p><h4 id="LVM-snapshot"><a href="#LVM-snapshot" class="headerlink" title="LVM snapshot"></a>LVM snapshot</h4><p>create a snapshot for annathin1</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# lvcreate -s -l 99 -n thinpool_snapshot /dev/thin_vg/annathin1</span><br><span class="line"> Using default stripesize 64.00 KiB.</span><br><span class="line"> WARNING: Sum of all thin volume sizes (10.00 GiB) exceeds the size of thin pools and the size of whole volume group (2.00 GiB)!</span><br><span class="line"> For thin pool auto extension activation/thin_pool_autoextend_threshold should be below 100.</span><br><span class="line"> Logical volume "thinpool_snapshot" created.</span><br><span class="line">[root@client ~]# lvdisplay /dev/thin_vg/thinpool_snapshot </span><br><span class="line"> --- Logical volume ---</span><br><span class="line"> LV Path /dev/thin_vg/thinpool_snapshot</span><br><span class="line"> LV Name thinpool_snapshot</span><br><span class="line"> VG Name thin_vg</span><br><span class="line"> LV UUID 4fYojx-SvHi-5IVX-aoIk-RxpD-8Nlb-1hHK3g</span><br><span class="line"> LV Write Access read/write</span><br><span class="line"> LV Creation host, time client.centos7.study, 2019-07-14 17:01:32 -0700</span><br><span class="line"> LV snapshot status active destination for annathin1</span><br><span class="line"> LV Status available</span><br><span class="line"> # open 0</span><br><span class="line"> LV Size 10.00 GiB</span><br><span class="line"> Current LE 2560</span><br><span class="line"> COW-table size 396.00 MiB</span><br><span class="line"> COW-table LE 99</span><br><span class="line"> Allocated to snapshot 0.02%</span><br><span class="line"> Snapshot chunk size 4.00 KiB</span><br><span class="line"> Segments 1</span><br><span class="line"> Allocation inherit</span><br><span class="line"> Read ahead sectors auto</span><br><span class="line"> - currently set to 8192</span><br><span class="line"> Block device 253:11</span><br><span class="line"></span><br><span class="line"></span><br><span class="line">[root@client ~]# lvdisplay /dev/thin_vg/annathin1 </span><br><span class="line"> --- Logical volume ---</span><br><span class="line"> LV Path /dev/thin_vg/annathin1</span><br><span class="line"> LV Name annathin1</span><br><span class="line"> VG Name thin_vg</span><br><span class="line"> LV UUID q47tG0-P2xf-CMsw-a2tE-SgWl-gjni-VJD1jA</span><br><span class="line"> LV Write Access read/write</span><br><span class="line"> LV Creation host, time client.centos7.study, 2019-07-12 13:09:12 -0700</span><br><span class="line"> LV snapshot status source of</span><br><span class="line"> thinpool_snapshot [active]</span><br><span class="line"> LV Pool name thinpool</span><br><span class="line"> LV Status available</span><br><span class="line"> # open 1</span><br><span class="line"> LV Size 10.00 GiB</span><br><span class="line"> Mapped size 0.12%</span><br><span class="line"> Current LE 2560</span><br><span class="line"> Segments 1</span><br><span class="line"> Allocation inherit</span><br><span class="line"> Read ahead sectors auto</span><br><span class="line"> - currently set to 8192</span><br><span class="line"> Block device 253:8</span><br></pre></td></tr></table></figure><p>Mount</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">[root@client /]# mkdir /mnt/snapshot</span><br><span class="line">[root@client snapshot]# blkid</span><br><span class="line">/dev/mapper/thin_vg-annathin1: UUID="93b48865-b28d-4e2e-b76e-893e804d944c" TYPE="xfs" </span><br><span class="line">/dev/mapper/thin_vg-thinpool_snapshot: UUID="93b48865-b28d-4e2e-b76e-893e804d944c" TYPE="xfs"</span><br><span class="line">[root@client /]# mount -o nouuid /dev/thin_vg/thinpool_snapshot /mnt/snapshot</span><br><span class="line">[root@client /]# df -Th</span><br><span class="line">Filesystem Type Size Used Avail Use% Mounted on</span><br><span class="line">/dev/mapper/thin_vg-annathin1 xfs 10G 34M 10G 1% /mnt/annathin1</span><br><span class="line">/dev/mapper/thin_vg-thinpool_snapshot xfs 10G 34M 10G 1% /mnt/snapshot</span><br></pre></td></tr></table></figure><p>Due to snapshot, /dev/thin_vg/annathin1 and /dev/thin_vg/thinpool_snapshot have same uuid. When mount to mount point, users need to use “nouuid” arguments.</p><h4 id="make-some-changes-on-the-mnt-annathin1-dir"><a href="#make-some-changes-on-the-mnt-annathin1-dir" class="headerlink" title="make some changes on the /mnt/annathin1 dir"></a>make some changes on the /mnt/annathin1 dir</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@client man-db-2.6.3]# rm ChangeLog</span><br><span class="line">rm: remove regular file ‘ChangeLog’?</span><br><span class="line">[root@client man-db-2.6.3]# df -Th /mnt/annathin1/ /mnt/snapshot/</span><br><span class="line">Filesystem Type Size Used Avail Use% Mounted on</span><br><span class="line">/dev/mapper/thin_vg-annathin1 xfs 10G 33M 10G 1% /mnt/annathin1</span><br><span class="line">/dev/mapper/thin_vg-thinpool_snapshot xfs 10G 34M 10G 1% /mnt/snapshot</span><br></pre></td></tr></table></figure><p>check the status of snapshot after doing the change on the original annathin1 dir</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line">[root@client man-db-2.6.3]# lvdisplay /dev/thin_vg/thinpool_snapshot </span><br><span class="line"> --- Logical volume ---</span><br><span class="line"> LV Path /dev/thin_vg/thinpool_snapshot</span><br><span class="line"> LV Name thinpool_snapshot</span><br><span class="line"> VG Name thin_vg</span><br><span class="line"> LV UUID eT1j03-4v5E-7fUi-BSV0-6dun-RCL1-fy9atx</span><br><span class="line"> LV Write Access read/write</span><br><span class="line"> LV Creation host, time client.centos7.study, 2019-07-14 17:18:16 -0700</span><br><span class="line"> LV snapshot status active destination for annathin1</span><br><span class="line"> LV Status available</span><br><span class="line"> # open 1</span><br><span class="line"> LV Size 10.00 GiB</span><br><span class="line"> Current LE 2560</span><br><span class="line"> COW-table size 396.00 MiB</span><br><span class="line"> COW-table LE 99</span><br><span class="line"> Allocated to snapshot 0.52%</span><br><span class="line"> Snapshot chunk size 4.00 KiB</span><br><span class="line"> Segments 1</span><br><span class="line"> Allocation inherit</span><br><span class="line"> Read ahead sectors auto</span><br><span class="line"> - currently set to 8192</span><br><span class="line"> Block device 253:11</span><br></pre></td></tr></table></figure><p>It shows the snapshot has been used 0.52%</p><p>next, we will restore annathin1 by snapshot, we will backup snapshot first.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# xfsdump -l 0 -L lvm1 -M lvm1 -f /tmp/backup.lvm /mnt/snapshot</span><br><span class="line">xfsdump: using file dump (drive_simple) strategy</span><br><span class="line">xfsdump: version 3.1.4 (dump format 3.0) - type ^C for status and control</span><br><span class="line">xfsdump: level 0 dump of client.centos7.study:/mnt/snapshot</span><br><span class="line">xfsdump: dump date: Sun Jul 14 17:31:38 2019</span><br><span class="line">xfsdump: session id: 5ea4e354-bf53-4069-8b63-ec4c2259c444</span><br><span class="line">xfsdump: session label: "lvm1"</span><br><span class="line">xfsdump: ino map phase 1: constructing initial dump list</span><br><span class="line">xfsdump: ino map phase 2: skipping (no pruning necessary)</span><br><span class="line">xfsdump: ino map phase 3: skipping (only one dump stream)</span><br><span class="line">xfsdump: ino map construction complete</span><br><span class="line">xfsdump: estimated dump size: 786816 bytes</span><br><span class="line">xfsdump: creating dump session media file 0 (media 0, file 0)</span><br><span class="line">xfsdump: dumping ino map</span><br><span class="line">xfsdump: dumping directories</span><br><span class="line">xfsdump: dumping non-directory files</span><br><span class="line">xfsdump: ending media file</span><br><span class="line">xfsdump: media file size 775272 bytes</span><br><span class="line">xfsdump: dump size (non-dir files) : 745816 bytes</span><br><span class="line">xfsdump: dump complete: 1 seconds elapsed</span><br><span class="line">xfsdump: Dump Summary:</span><br><span class="line">xfsdump: stream 0 /tmp/backup.lvm OK (success)</span><br><span class="line">xfsdump: Dump Status: SUCCESS</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br></pre></td><td class="code"><pre><span class="line">[root@client ~]# umount /mnt/snapshot</span><br><span class="line">[root@client ~]# lvremove /dev/thin_vg/thinpool_snapshot </span><br><span class="line">Do you really want to remove active logical volume thin_vg/thinpool_snapshot? [y/n]: y</span><br><span class="line"> Logical volume "thinpool_snapshot" successfully removed</span><br><span class="line">[root@client ~]# umount /mnt/annathin1/</span><br><span class="line">[root@client ~]# mkfs.xfs -f /dev/thin_vg/annathin1 </span><br><span class="line">meta-data=/dev/thin_vg/annathin1 isize=512 agcount=16, agsize=163824 blks</span><br><span class="line"> = sectsz=512 attr=2, projid32bit=1</span><br><span class="line"> = crc=1 finobt=0, sparse=0</span><br><span class="line">data = bsize=4096 blocks=2621184, imaxpct=25</span><br><span class="line"> = sunit=16 swidth=16 blks</span><br><span class="line">naming =version 2 bsize=4096 ascii-ci=0 ftype=1</span><br><span class="line">log =internal log bsize=4096 blocks=2560, version=2</span><br><span class="line"> = sectsz=512 sunit=16 blks, lazy-count=1</span><br><span class="line">realtime =none extsz=4096 blocks=0, rtextents=0</span><br><span class="line">[root@client ~]# mount /dev/thin_vg/annathin1 /mnt/annathin1/</span><br><span class="line"></span><br><span class="line">[root@client ~]# xfsrestore -f /tmp/backup.lvm -L lvm1 /mnt/annathin1/</span><br><span class="line">xfsrestore: using file dump (drive_simple) strategy</span><br><span class="line">xfsrestore: version 3.1.4 (dump format 3.0) - type ^C for status and control</span><br><span class="line">xfsrestore: searching media for dump</span><br><span class="line">xfsrestore: examining media file 0</span><br><span class="line">xfsrestore: found dump matching specified label:</span><br><span class="line">xfsrestore: hostname: client.centos7.study</span><br><span class="line">xfsrestore: mount point: /mnt/snapshot</span><br><span class="line">xfsrestore: volume: /dev/mapper/thin_vg-thinpool_snapshot</span><br><span class="line">xfsrestore: session time: Sun Jul 14 17:31:38 2019</span><br><span class="line">xfsrestore: level: 0</span><br><span class="line">xfsrestore: session label: "lvm1"</span><br><span class="line">xfsrestore: media label: "lvm1"</span><br><span class="line">xfsrestore: file system id: 93b48865-b28d-4e2e-b76e-893e804d944c</span><br><span class="line">xfsrestore: session id: 5ea4e354-bf53-4069-8b63-ec4c2259c444</span><br><span class="line">xfsrestore: media id: e223f10b-7718-4801-bda7-7e066e59deb1</span><br><span class="line">xfsrestore: using online session inventory</span><br><span class="line">xfsrestore: searching media for directory dump</span><br><span class="line">xfsrestore: reading directories</span><br><span class="line">xfsrestore: 3 directories and 13 entries processed</span><br><span class="line">xfsrestore: directory post-processing</span><br><span class="line">xfsrestore: restoring non-directory files</span><br><span class="line">xfsrestore: restore complete: 0 seconds elapsed</span><br><span class="line">xfsrestore: Restore Summary:</span><br><span class="line">xfsrestore: stream 0 /tmp/backup.lvm OK (success)</span><br><span class="line">xfsrestore: Restore Status: SUCCESS</span><br><span class="line"></span><br><span class="line">[root@client ~]# df -Th /mnt/annathin1/</span><br><span class="line">Filesystem Type Size Used Avail Use% Mounted on</span><br><span class="line">/dev/mapper/thin_vg-annathin1 xfs 10G 34M 10G 1% /mnt/annathin1</span><br></pre></td></tr></table></figure><p>It shows the annathins dir back from 33M to 34M, the original one.</p>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Cyber Security: Risk Management Report</title>
<link href="/2019/02/09/Cyber-Security-Risk-Management-Report/"/>
<url>/2019/02/09/Cyber-Security-Risk-Management-Report/</url>
<content type="html"><![CDATA[<p>Identify the potential risk or threat on computer\iPad\Phone, decide how the impact and likely of threat, deal the threat with the mitigation, using the risk chart to record the steps.</p><h4 id="Item-1-name-ThinkPad-laptop-2017"><a href="#Item-1-name-ThinkPad-laptop-2017" class="headerlink" title="Item 1 name: ThinkPad laptop 2017"></a>Item 1 name: ThinkPad laptop 2017</h4><p>Description: This is a laptop with a lot of important files on it.<br>Threat 1: Virus Attacks that can damage computer systems and data.<br>Mitigation: Install the anti-virus software, such as McAfee.</p><h4 id="Item-2-name-laptop-hard-drive"><a href="#Item-2-name-laptop-hard-drive" class="headerlink" title="Item 2 name: laptop hard drive"></a>Item 2 name: laptop hard drive</h4><p>Description: Hard drive is the most important part in laptop. With the damage hard drive, the laptop can’t even start.<br>Threat 2: HDD has unrecoverable failure and lost all data on laptop<br>Mitigation: Make extra copy on other computer or buy extra HDD for storing important files.</p><a id="more"></a><h4 id="Item-3-name-host-station"><a href="#Item-3-name-host-station" class="headerlink" title="Item 3 name: host station"></a>Item 3 name: host station</h4><p>Description: It should be a firewall between the internal and external net in order to keep the host station safe.<br>Threat 3: Internet Hacks<br>Mitigation: Placed a firewall between trusted network and the untrusted network.</p><h4 id="Item-4-name-iPhone"><a href="#Item-4-name-iPhone" class="headerlink" title="Item 4 name: iPhone"></a>Item 4 name: iPhone</h4><p>Description: having a lot of important information on it, such as credit card, PayPal account, BOA bank account, it will be the target of malicious program by the third party.<br>Threat 4: The malicious third-party thief the information.<br>Mitigation: Keep the operating system and application update all the time and use encryption.</p><h4 id="Item-5-name-iPad"><a href="#Item-5-name-iPad" class="headerlink" title="Item 5 name: iPad"></a>Item 5 name: iPad</h4><p>Description: iPad uses Wi-Fi to connect to internet and is easy to be hacked.<br>Threat 5: Wireless hacker.<br>Mitigation: take penetration test. </p>]]></content>
<tags>
<tag> cyber security </tag>
</tags>
</entry>
<entry>
<title>Python-using repetition structures nest loop to print patterns</title>
<link href="/2019/01/08/Python-using-repetition-structures-to-draw-mode/"/>
<url>/2019/01/08/Python-using-repetition-structures-to-draw-mode/</url>
<content type="html"><![CDATA[<h4 id="Two-repetition-structures-in-python"><a href="#Two-repetition-structures-in-python" class="headerlink" title="Two repetition structures in python:"></a>Two repetition structures in python:</h4><ul><li>while loop</li><li>for loop<h4 id="Nest-loop"><a href="#Nest-loop" class="headerlink" title="Nest loop"></a>Nest loop</h4>For loop has a nest loop function. clock is a good example.<br>Nest loop can use to print some patterns.<br>Some examples:</li></ul><h6 id="pattern-1"><a href="#pattern-1" class="headerlink" title="pattern 1"></a>pattern 1</h6><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">rows = int(input('how many rows?'))</span><br><span class="line">cols = int(input('how many columns?'))</span><br><span class="line">for r in range(rows):</span><br><span class="line"> for c in range(cols):</span><br><span class="line"> print('*',end='')</span><br><span class="line"> print()</span><br></pre></td></tr></table></figure><p>after input rows and clos value 6, runing code, it will show</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">******</span><br><span class="line">******</span><br><span class="line">******</span><br><span class="line">******</span><br><span class="line">******</span><br><span class="line">******</span><br></pre></td></tr></table></figure><h6 id="pattern-2"><a href="#pattern-2" class="headerlink" title="pattern 2"></a>pattern 2</h6><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">*</span><br><span class="line">**</span><br><span class="line">***</span><br><span class="line">****</span><br><span class="line">*****</span><br><span class="line">******</span><br><span class="line">*******</span><br></pre></td></tr></table></figure><p>code as following:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">base_size = 7</span><br><span class="line">for r in range(base_size):</span><br><span class="line"> for c in range(r+1):</span><br><span class="line"> print('*',end='')</span><br><span class="line"> print()</span><br></pre></td></tr></table></figure><h6 id="pattern-3-up-side-down-triangle"><a href="#pattern-3-up-side-down-triangle" class="headerlink" title="pattern 3: up side down triangle:"></a>pattern 3: up side down triangle:</h6><a id="more"></a><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">*******</span><br><span class="line">******</span><br><span class="line">*****</span><br><span class="line">****</span><br><span class="line">***</span><br><span class="line">**</span><br><span class="line">*</span><br></pre></td></tr></table></figure><p>code like this:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">rows =7</span><br><span class="line">cols =7</span><br><span class="line"></span><br><span class="line">for i in range(rows):</span><br><span class="line"> #num_of_star = cols - i</span><br><span class="line"> #print('*' * (cols - i))</span><br><span class="line"> for j in range(cols - i):</span><br><span class="line"> print('*', end='')</span><br><span class="line"> print()</span><br></pre></td></tr></table></figure><h6 id="pattern-4"><a href="#pattern-4" class="headerlink" title="pattern 4:"></a>pattern 4:</h6><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">*</span><br><span class="line"> *</span><br><span class="line"> *</span><br><span class="line"> *</span><br><span class="line"> *</span><br><span class="line"> *</span><br></pre></td></tr></table></figure><p>code for this pattern is:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">num_step =6</span><br><span class="line">for r in range(num_steps):</span><br><span class="line"> for c in range(r):</span><br><span class="line"> print(' ',end='')</span><br><span class="line"> print('#')</span><br></pre></td></tr></table></figure><h6 id="pattern-5"><a href="#pattern-5" class="headerlink" title="pattern 5"></a>pattern 5</h6><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">**</span><br><span class="line">* *</span><br><span class="line">* *</span><br><span class="line">* *</span><br><span class="line">* *</span><br><span class="line">* *</span><br></pre></td></tr></table></figure><p>code for this patter is:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">col = 6</span><br><span class="line">raw = 6</span><br><span class="line">for a in range(raw):</span><br><span class="line"> print('#',end='', sep='')</span><br><span class="line"> for b in range(a):</span><br><span class="line"> print(' ', end='', sep='')</span><br><span class="line"> print('#')</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> python </tag>
</tags>
</entry>
<entry>
<title>Cyber Security: Security Audit Report about Computer System</title>
<link href="/2018/07/20/Cyber-Security-Security-Audit-Report-about-Computer-System/"/>
<url>/2018/07/20/Cyber-Security-Security-Audit-Report-about-Computer-System/</url>
<content type="html"><![CDATA[<h4 id="Abstract"><a href="#Abstract" class="headerlink" title="Abstract:"></a>Abstract:</h4><p>Using MBSA to audit computer to find 4 things need to be fixed, some missing update, some passwords are vulnerable. To fix these problems, update application and change the password policy to force users to change passwords.</p><h3 id="Methods"><a href="#Methods" class="headerlink" title="Methods:"></a>Methods:</h3><p>Got MBSA App 2.1 from Microsoft website and install it.</p><p>Get started to scam computer, it takes a little longer than expected to finish scan.</p><h3 id="Finding-1"><a href="#Finding-1" class="headerlink" title="Finding 1:"></a>Finding 1:</h3><p>Description: The Automatic Updates system service is not configured to be started as Automatic.</p><p>Mitigation: Since sometimes automatic updates is not always good to computer system and network, it is not necessary to update all the time. Keep this configuration not change, then regularly check the system service to determine which needs to be updated.</p><h3 id="Finding-2"><a href="#Finding-2" class="headerlink" title="Finding 2:"></a>Finding 2:</h3><p>Description: Some user accounts (4 of 6) have blank or simple passwords, or could not be analyzed.</p><p>Mitigation: change password policy and inforce the password security, let the users change their passwords.</p><a id="more"></a><h3 id="Finding-3"><a href="#Finding-3" class="headerlink" title="Finding 3:"></a>Finding 3:</h3><p>Description: a Virtual Box software update installation was not completed.</p><p>Mitigation: restart the computer immediately following the installation of security updates, both for protection and stability of the system.</p><h3 id="Finding-4"><a href="#Finding-4" class="headerlink" title="Finding 4:"></a>Finding 4:</h3><p>Description: Some user accounts (4 of 6) have non-expiring passwords</p><p>Mitigation: Set the password expiring for the users.</p><p>Audit plan:</p><img src="/2018/07/20/Cyber-Security-Security-Audit-Report-about-Computer-System/audit.jpg" class=""><p>Citations:<br>Cybersecurity Analyst+: Scan for Vulnerabilities using Microsoft Baseline Security Analyzer By: Dan Lachance<br>Microsoft Security Fundamentals: Microsoft Baseline Security Analyzer (MBSA) By: Travis Welton<br>Baseline Security Evaluation Checklist <a href="acm.org">acm.org</a></p>]]></content>
<tags>
<tag> cyber security </tag>
</tags>
</entry>
<entry>
<title>Cyber Security Policy Goals of Chabot College</title>
<link href="/2018/07/11/Cyber-Security-Policy-Goals-of-Chabot-College/"/>
<url>/2018/07/11/Cyber-Security-Policy-Goals-of-Chabot-College/</url>
<content type="html"><![CDATA[<h4 id="Clean-Desk-Policy"><a href="#Clean-Desk-Policy" class="headerlink" title="Clean Desk Policy:"></a>Clean Desk Policy:</h4><h3 id="Goal"><a href="#Goal" class="headerlink" title="Goal:"></a>Goal:</h3><p>Establish the minimum requirements for maintaining a “clean desk” – where sensitive/critical information about employees, intellectual property, students is secure in locked areas and out of site.</p><h3 id="Procedure"><a href="#Procedure" class="headerlink" title="Procedure:"></a>Procedure:</h3><p> a.Employees are required to ensure that all sensitive/confidential information in hardcopy or electronic form is secure in their work area at the end of the day and when they are expected to be gone for an extended period.<br> b.Computer workstations must be locked when workspace is unoccupied.<br> c.Computer workstations must be shut completely down at the end of the work day.<br> d.Any Restricted or Sensitive information must be removed from the desk and locked in a drawer when the desk is unoccupied and at the end of the work day.<br> e.File cabinets containing Restricted or Sensitive information must be kept closed and locked when not in use or when not attended.<br> f.Keys used for access to Restricted or Sensitive information must not be left at an unattended desk.<br> g.Laptops must be either locked with a locking cable or locked away in a drawer.<br> h.Passwords may not be left on sticky notes posted on or under a computer, nor may they be left written down in an accessible location.<br> i.Printouts containing Restricted or Sensitive information should be immediately removed from the printer.<br> j.Upon disposal Restricted and/or Sensitive documents should be shredded in the official shredder bins or placed in the lock confidential disposal bins.<br> k.Whiteboards containing Restricted and/or Sensitive information should be erased.<br> l.Lock away portable computing devices such as laptops and tablets.<br> m.Treat mass storage devices such as CDROM, DVD or USB drives as sensitive and secure them in a locked drawer.</p><h4 id="Password-Policy"><a href="#Password-Policy" class="headerlink" title="Password Policy:"></a>Password Policy:</h4><h3 id="Goal-1"><a href="#Goal-1" class="headerlink" title="Goal:"></a>Goal:</h3><p> Passwords are a critical component of information security and serve to protect user accounts; however, a poorly constructed password may result in the compromise of individual systems, data, or network. This policy provides best practices for creating secure passwords.</p><a id="more"></a><h3 id="Procedure-1"><a href="#Procedure-1" class="headerlink" title="Procedure:"></a>Procedure:</h3><p>a.Strong passwords are long, the more characters you have the stronger the password.<br>b. We recommend a minimum of 14 characters in your password.<br>c. We highly encourage the use of passphrases, encourage the use of ‘password manager’ software,passwords made up of multiple words.</p><h4 id="Email-Policy"><a href="#Email-Policy" class="headerlink" title="Email Policy:"></a>Email Policy:</h4><h3 id="Goal-2"><a href="#Goal-2" class="headerlink" title="Goal:"></a>Goal:</h3><p>The purpose of this policy is to ensure the proper use of email system and make users aware of Chabot College deems as acceptable and unacceptable use of its email system. This policy outlines the minimum requirements for use of email within Chabot College Network.</p><h3 id="Procedure-2"><a href="#Procedure-2" class="headerlink" title="Procedure:"></a>Procedure:</h3><p>a. All use of email must be consistent with Chabot College policies and procedures of ethical conduct, safety, compliance with applicable laws and proper business practices.<br>b. Email account should be used primarily for Chabot College business-related purposes; personal communication is permitted on a limited basis.<br>c. All Chabot College data contained within an email message or an attachment must be secured according to the Data Protection Standard.<br>d.The Chabot College email system shall not to be used for the creation or distribution of any disruptive or offensive messages, including offensive comments about race, gender, hair color, disabilities, age, sexual orientation, pornography, religious beliefs and practice, political beliefs, or national origin.</p><p>Related Standards, Policies and Processes:<br>Data Protection Standard</p><h4 id="End-User-Encryption-Key-Protection-Policy"><a href="#End-User-Encryption-Key-Protection-Policy" class="headerlink" title="End User Encryption Key Protection Policy:"></a>End User Encryption Key Protection Policy:</h4><h3 id="Goal-3"><a href="#Goal-3" class="headerlink" title="Goal:"></a>Goal:</h3><p>This policy outlines the requirements for protecting encryption keys that are under the control of end users. These requirements are designed to prevent unauthorized disclosure and subsequent fraudulent use. The protection methods outlined will include operational and technical controls, such as key backup procedures, encryption under a separate key and use of tamper-resistant hardware.</p><h3 id="Procedure-3"><a href="#Procedure-3" class="headerlink" title="Procedure:"></a>Procedure:</h3><p>a.Keys used for secret key encryption, also called symmetric cryptography, must be protected as they are distributed to all parties that will use them. During distribution, the symmetric encryption keys must be encrypted using a stronger algorithm with a key of the longest key length for that algorithm authorized in Chabot College’s Acceptable Encryption Policy. If the keys are for the strongest algorithm, then the key must be split, each portion of the key encrypted with a different key that is the longest key length authorized and each encrypted portion is transmitted using different transmission mechanisms. The goal is to provide more stringent protection to the key than the data that is encrypted with that encryption key.</p><p>b.Public key cryptography, or asymmetric cryptography, uses public-private key pairs. The public key is passed to the certificate authority to be included in the digital certificate issued to the end user. The digital certificate is available to everyone once it issued. The private key should only be available to the end user to whom the corresponding digital certificate is issued.</p><p>c.Hardware tokens storing encryption keys will be treated as sensitive equipment, as described in Chabot College’s Physical Security policy. In addition, all hardware tokens, smartcards, USB tokens, etc.will not be stored or left connected to any end user’s computer when not in use. For end users traveling with hardware tokens, they will not be stored or carried in the same container or bag as any computer.</p><p>d.All PINs, passwords or passphrases used to protect encryption keys must meet complexity and length requirements described in Chabot College’s Password Policy.</p><p>e.The Infosec team will verify compliance to this policy through various methods, including but not limited to, periodic walkthroughs, video monitoring, business tool reports, internal and external audits, and feedback to the policy owner.</p><p>Related Standards, Policies and Processes:<br> a.Password Policy<br> b.Encryption Policy</p><h4 id="Lab-Anti-Virus-Policy"><a href="#Lab-Anti-Virus-Policy" class="headerlink" title="Lab Anti-Virus Policy:"></a>Lab Anti-Virus Policy:</h4><h3 id="Goal-4"><a href="#Goal-4" class="headerlink" title="Goal:"></a>Goal:</h3><p> To ensure effective virus detection and prevention in Chabot College lab.</p><h3 id="Procedure-4"><a href="#Procedure-4" class="headerlink" title="Procedure:"></a>Procedure:</h3><p>a. All Chabot College PC-based lab computers must have Chabot College’s standard, supported anti-virus software installed and scheduled to run at regular intervals. In addition, the anti-virus software and the virus pattern files must be kept up-to-date. b.Virus-infected computers must be removed from the network until they are verified as virus-free.<br>b.Lab Admins/Lab Managers are responsible for creating procedures that ensure anti-virus software is run at regular intervals, and computers are verified as virus-free. d.Any activities with the intention to create and/or distribute malicious programs into Chabot College’s networks (e.g., viruses, worms, Trojan horses, e-mail bombs, etc.) are prohibited, in accordance with the Acceptable Use Policy.</p><p>Related Standards, Policies and Processes:<br>•Acceptable Use Policy<br>•Anti-Virus Recommended Processes<br>The CmapTools Mind:</p><img src="/2018/07/11/Cyber-Security-Policy-Goals-of-Chabot-College/labantivirus.jpg" class=""><p>Ganttproject timeline:</p><img src="/2018/07/11/Cyber-Security-Policy-Goals-of-Chabot-College/gantt.jpg" class=""><p>Citation:<a href="https://www.sans.org/security-resources/policies" target="_blank" rel="noopener"> https://www.sans.org/security-resources/policies</a></p>]]></content>
<tags>
<tag> cyber security </tag>
</tags>
</entry>
<entry>
<title>Linux: ACL, useradd and groupadd</title>
<link href="/2018/04/02/Linux-ACL-useradd-and-groupadd/"/>
<url>/2018/04/02/Linux-ACL-useradd-and-groupadd/</url>
<content type="html"><![CDATA[<p>ACL means “access control list”.</p><h4 id="Config-file"><a href="#Config-file" class="headerlink" title="Config file"></a>Config file</h4><p>/etc/default/useradd<br>/etc/skel<br>note:this config will add subdir under /etc/skel, it will automatically create the same dir when using useradd command to create a username</p><h4 id="useradd"><a href="#useradd" class="headerlink" title="useradd"></a>useradd</h4><p>-u uid<br>-g default group, initial login group<br>-a -G append extra group<br>-r add a system users (uid < 100)<br>-s shell /bin/bash or /sbin/login or /sbin/nologin<br>newgrp username groupname set the effective group for user</p><h4 id="userdel"><a href="#userdel" class="headerlink" title="userdel"></a>userdel</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">userdel anna</span><br></pre></td></tr></table></figure><p>only delete the username, keep the home dir and all other files</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">userdel -r anna</span><br></pre></td></tr></table></figure><p>delete completely all the username’s file and dir, use “-r”</p><h4 id="gpasswd"><a href="#gpasswd" class="headerlink" title="gpasswd"></a>gpasswd</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gpasswd group name</span><br></pre></td></tr></table></figure><p>create a passwd for this group</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">gpasswd -A username groupname</span><br></pre></td></tr></table></figure><p>Let username to be the administrator of groupname<br>Then user can use gpasswd to add new user to this group</p><a id="more"></a><h4 id="Access-control-list"><a href="#Access-control-list" class="headerlink" title="Access control list"></a>Access control list</h4><p>Aim to:<br>users not the owner of file<br>groups not the group of file<br>specific to file or dir</p><p>check the file or dir’s acl status</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">getfacl filename</span><br><span class="line">getfacl dirname</span><br></pre></td></tr></table></figure><p>let user anna has rx rights to file /etc/shadow<br>“d:” means let anna has rx rights all the way down to /srv dir</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">setfacl -m u:anna:rx /etc/shadow</span><br><span class="line">setfacl -m d:u:anna:rx /srv</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">setfacl -x u:anna /etc/shadow</span><br><span class="line">setfacl -x d:u:anna /srv</span><br></pre></td></tr></table></figure><p>delete user anna rights from ACL, doesn’t need to specific the rwx</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">setfacl -b filename</span><br><span class="line">setfacl -b dir</span><br></pre></td></tr></table></figure><p>delete all acl config from file or dir</p><h4 id="Some-useful-command-to-check-the-state-of-id-or-password"><a href="#Some-useful-command-to-check-the-state-of-id-or-password" class="headerlink" title="Some useful command to check the state of id or password"></a>Some useful command to check the state of id or password</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">id</span><br><span class="line">groups</span><br></pre></td></tr></table></figure><p>check the password</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">pwck</span><br></pre></td></tr></table></figure><p>check the group status</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">grpck</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Linux: some very useful commands of rpm</title>
<link href="/2018/03/29/Linux-some-very-useful-commands-of-rpm/"/>
<url>/2018/03/29/Linux-some-very-useful-commands-of-rpm/</url>
<content type="html"><![CDATA[<p>RPM (Red Hat Package Manager) is an default open source and most popular package management utility in Redhat Linux. The tool allows users to install, update, uninstall, query, verify and manage system software packages in Unix/Linux operating systems.<br>Here are some very useful commands</p><h4 id="You-want-to-know-whether-httpd-installed"><a href="#You-want-to-know-whether-httpd-installed" class="headerlink" title="You want to know whether httpd installed"></a>You want to know whether httpd installed</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@www 7]# rpm -qa |grep httpd</span><br><span class="line">httpd-tools-2.4.6-88.el7.centos.x86_64</span><br><span class="line">httpd-2.4.6-88.el7.centos.x86_64</span><br></pre></td></tr></table></figure><h4 id="Query-all-the-config-files-of-http"><a href="#Query-all-the-config-files-of-http" class="headerlink" title="Query all the config files of http"></a>Query all the config files of http</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line">[root@www 7]# rpm -qc httpd</span><br><span class="line">/etc/httpd/conf.d/autoindex.conf</span><br><span class="line">/etc/httpd/conf.d/userdir.conf</span><br><span class="line">/etc/httpd/conf.d/welcome.conf</span><br><span class="line">/etc/httpd/conf.modules.d/00-base.conf</span><br><span class="line">/etc/httpd/conf.modules.d/00-dav.conf</span><br><span class="line">/etc/httpd/conf.modules.d/00-lua.conf</span><br><span class="line">/etc/httpd/conf.modules.d/00-mpm.conf</span><br><span class="line">/etc/httpd/conf.modules.d/00-proxy.conf</span><br><span class="line">/etc/httpd/conf.modules.d/00-systemd.conf</span><br><span class="line">/etc/httpd/conf.modules.d/01-cgi.conf</span><br><span class="line">/etc/httpd/conf/httpd.conf</span><br><span class="line">/etc/httpd/conf/magic</span><br><span class="line">/etc/logrotate.d/httpd</span><br><span class="line">/etc/sysconfig/htcacheclean</span><br><span class="line">/etc/sysconfig/httpd</span><br></pre></td></tr></table></figure><h4 id="Query-all-the-files-about-httpd"><a href="#Query-all-the-files-about-httpd" class="headerlink" title="Query all the files about httpd"></a>Query all the files about httpd</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">rpm -ql httpd</span><br></pre></td></tr></table></figure><h4 id="If-you-don’t-know-which-software-a-file-belongs-to-you-can-do"><a href="#If-you-don’t-know-which-software-a-file-belongs-to-you-can-do" class="headerlink" title="If you don’t know which software a file belongs to, you can do :"></a>If you don’t know which software a file belongs to, you can do :</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[root@www 7]# rpm -qf /etc/rpc</span><br><span class="line">glibc-2.17-260.el7_6.3.x86_64</span><br><span class="line">glibc-2.17-260.el7_6.3.i686</span><br></pre></td></tr></table></figure><a id="more"></a><h4 id="Show-all-changes-have-been-made-after-you-install-a-package"><a href="#Show-all-changes-have-been-made-after-you-install-a-package" class="headerlink" title="Show all changes have been made after you install a package."></a>Show all changes have been made after you install a package.</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[root@www 7]# rpm -Va</span><br><span class="line">..5...... /usr/lib/firmware/vpu_d.bin</span><br><span class="line">S.5....T. c /etc/yum/pluginconf.d/langpacks.conf</span><br><span class="line">S.5....T. c /etc/ssh/sshd_config</span><br><span class="line">S.5....T. c /etc/sysctl.conf</span><br><span class="line">.M....... g /var/log/dmesg</span><br></pre></td></tr></table></figure><p>S T c g have different meanings</p><p>S ：(file Size differs)<br>M ：(Mode differs)<br>5 ：(MD5 sum differs)<br>D ：(Device major/minor number mis-match)<br>L ：(readLink(2) path mis-match)<br>U ：(User ownership differs)<br>G ：(Group ownership differs)<br>T ：(mTime differs)<br>c ：(config file)<br>d ：(documentation)<br>g ：(ghost file)<br>l ：(license file)<br>r ：(read me)<br>###To be specific, you want to know whether special file was changed</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">[root@www 7]# rpm -Vf /etc/default/grub </span><br><span class="line">S.5....T. c /etc/grub.d/40_custom</span><br></pre></td></tr></table></figure><p>It shows /etc/default/grub has been changed S(size) 5 (MD5 sum differs), T (Mtime ).</p>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Linux: some useful command of yum</title>
<link href="/2018/03/19/Linux-some-useful-command-of-yum/"/>
<url>/2018/03/19/Linux-some-useful-command-of-yum/</url>
<content type="html"><![CDATA[<p>Yum is a very powful tool in Redhat Linux, it can provide install and update software online.</p><h4 id="Some-very-useful-commands"><a href="#Some-very-useful-commands" class="headerlink" title="Some very useful commands"></a>Some very useful commands</h4><p>Quiry</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum list all</span><br></pre></td></tr></table></figure><p>query all softwares in repository</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum list installed</span><br></pre></td></tr></table></figure><p>softwares already installed on host</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum list available</span><br></pre></td></tr></table></figure><p>softwares not installed yet, you can install right now</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum list updates</span><br></pre></td></tr></table></figure><p>show the software that can be updated at once</p><p>The following two commands have same function</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">yum provides passwd</span><br><span class="line">rpm -qf /etc/passwd</span><br></pre></td></tr></table></figure><p>Both will tell you what software this command belong to</p><a id="more"></a><h4 id="Yum-config-documents-location"><a href="#Yum-config-documents-location" class="headerlink" title="Yum config documents location"></a>Yum config documents location</h4><p>/etc/yum.repo.d/<br>/etc/yum.conf</p><h4 id="About-groupinstall"><a href="#About-groupinstall" class="headerlink" title="About groupinstall"></a>About groupinstall</h4><p>If you want to install a group software, for example,</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">yum groupinfo "Scientific Support"</span><br><span class="line">Yum groupinstall "Scientific Support"</span><br></pre></td></tr></table></figure><p>It will show error</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">Loaded plugins: fastestmirror, langpacks</span><br><span class="line">Loading mirror speeds from cached hostfile</span><br><span class="line"> * base: mirror.fileplanet.com</span><br><span class="line"> * extras: mirror.fileplanet.com</span><br><span class="line"> * updates: mirror.fileplanet.com</span><br><span class="line">base | 3.6 kB 00:00:00 </span><br><span class="line">extras | 3.4 kB 00:00:00 </span><br><span class="line">ftp.ksu.edu.tw_FTP_CentOS_7_os_x86_64_ | 3.6 kB 00:00:00 </span><br><span class="line">updates | 3.4 kB 00:00:00 </span><br><span class="line">Warning: Group scientific does not have any packages to install.</span><br><span class="line">Maybe run: yum groups mark install (see man yum)</span><br><span class="line">No packages in any requested group available to install or update</span><br></pre></td></tr></table></figure><p>You need to add “group_package_types=default,mandatory,optional” to configuration file.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim /etc/yum.conf</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">group_package_types=default,mandatory,optional</span><br></pre></td></tr></table></figure><p>After doing that, you can install the group software successfully.</p><h4 id="Add-baseurl-to-your-yum-repository"><a href="#Add-baseurl-to-your-yum-repository" class="headerlink" title="Add baseurl to your yum repository"></a>Add baseurl to your yum repository</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum-config-manager --add-repo=http://www.centos.ann</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[centos.ann]</span><br><span class="line">name=added from: http://www.centos.ann</span><br><span class="line">baseurl=http://www.centos.ann</span><br><span class="line">enabled=1</span><br></pre></td></tr></table></figure><p>It will appear to list /var/cache/yum/x86_64/7</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cd /var/cache/yum/x86_64/7</span><br><span class="line">ll</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">drwxr-xr-x. 4 root root 33 Mar 8 11:13 centos.ann</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum repolist</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">repo id repo name status</span><br><span class="line">base/7/x86_64 CentOS-7 - Base 10,019</span><br><span class="line">centos.ann added from: http://www.centos.ann 0</span><br></pre></td></tr></table></figure><p>It shows centos.ann is the default yum base. 0 means there is no software in it.</p><h4 id="Use-local-disk-or-CD-to-yum-software"><a href="#Use-local-disk-or-CD-to-yum-software" class="headerlink" title="Use local disk or CD to yum software"></a>Use local disk or CD to yum software</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">vim /etc/yum.repos.d/cdrom.repo</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">[CDROM]</span><br><span class="line">name=cdrom</span><br><span class="line">baseurl=file:///mnt</span><br><span class="line">enable=1</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">yum --enablerepo=cdrom</span><br></pre></td></tr></table></figure><p>In case the rpm verify, you need to add pki</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">rpm --import /etc/pki/rpm-gpg/RPM-GPG-KEY-CentOS-7</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Linux: Create a swap</title>
<link href="/2018/03/15/Linux-Create-a-swap/"/>
<url>/2018/03/15/Linux-Create-a-swap/</url>
<content type="html"><![CDATA[<p>There are two ways to create a swap in Linux.</p><h4 id="Use-disk-partition"><a href="#Use-disk-partition" class="headerlink" title="Use disk partition"></a>Use disk partition</h4><p>Make a partition first</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">fdisk /dev/sda1</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">mkswap /dev/sda1</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">swapon /dev/sda1</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">blkid</span><br><span class="line">UUID="40a10328-63e5-4450-8bed-1739bc472d2d"</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vi /etc/fstab</span><br><span class="line">UUID="40a10328-63e5-4450-8bed-1739bc472d2d" swap swap defaults 0 0</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">mount -a</span><br></pre></td></tr></table></figure><h4 id="Use-a-file"><a href="#Use-a-file" class="headerlink" title="Use a file"></a>Use a file</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">dd if=/dev/zero of=/tmp/swap bs=1M count=128</span><br><span class="line">mkswap /tmp/swap</span><br><span class="line">swapon /tmp/swap</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">vi /etc/fstab</span><br><span class="line">/tmp/swap swap swap defaults 0 0</span><br></pre></td></tr></table></figure><hr>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Linux: config Grub2, adding more kernel menu, personality environment</title>
<link href="/2018/03/08/Linux-Config-grub2/"/>
<url>/2018/03/08/Linux-Config-grub2/</url>
<content type="html"><![CDATA[<p>Linux provides a method to let users create personality boot menu.Doing as follows:</p><h4 id="Config-document-location"><a href="#Config-document-location" class="headerlink" title="Config document location"></a>Config document location</h4><p>/boot/grub2/grub.cfg<br>/etc/default/grub<br>/etc/grub.d/*</p><h4 id="Adding-more-than-one-kernel-boot-to-menu"><a href="#Adding-more-than-one-kernel-boot-to-menu" class="headerlink" title="Adding more than one kernel boot to menu"></a>Adding more than one kernel boot to menu</h4><p>Copy first menu from /boot/grub2/grub.cfg</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">menuentry 'CentOS Linux (3.10.0-957.10.1.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted $menuentry_id_option 'gnulinux-3.10.0-957.10.1.el7.x86_64-advanced-51497437-886c-472c-9c74-8ef2e459214b' {</span><br><span class="line"> load_video</span><br><span class="line"> set gfxpayload=keep</span><br><span class="line"> insmod gzio</span><br><span class="line"> insmod part_msdos</span><br><span class="line"> insmod xfs</span><br><span class="line"> set root='hd0,msdos1'</span><br><span class="line"> if [ x$feature_platform_search_hint = xy ]; then</span><br><span class="line"> search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' cdbb3f80-2c48-4c5c-bc72-0f40023061b9</span><br><span class="line"> else</span><br><span class="line"> search --no-floppy --fs-uuid --set=root cdbb3f80-2c48-4c5c-bc72-0f40023061b9</span><br><span class="line"> fi</span><br><span class="line"> linux16 /vmlinuz-3.10.0-957.10.1.el7.x86_64 root=/dev/mapper/cl-root ro crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap ifname=enp0s3:08:00:27:37:8e:ba elevator=deadline elevator=deadline</span><br><span class="line"> initrd16 /initramfs-3.10.0-957.10.1.el7.x86_64.img</span><br><span class="line">}</span><br></pre></td></tr></table></figure><p>then</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">cd /etc/grub.d</span><br><span class="line">vim 40_custom</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line">menuentry 'my CentOS Linux (3.10.0-957.5.1.el7.x86_64) 7 (Core)' --class centos --class gnu-linux --class gnu --class os --unrestricted --id 'mygraphical' {</span><br><span class="line"> load_video</span><br><span class="line"> set gfxpayload=keep</span><br><span class="line"> insmod gzio</span><br><span class="line"> insmod part_msdos</span><br><span class="line"> insmod xfs</span><br><span class="line"> set root='hd0,msdos1'</span><br><span class="line"> if [ x$feature_platform_search_hint = xy ]; then</span><br><span class="line"> search --no-floppy --fs-uuid --set=root --hint-bios=hd0,msdos1 --hint-efi=hd0,msdos1 --hint-baremetal=ahci0,msdos1 --hint='hd0,msdos1' cdbb3f80-2c48-4c5c-bc72-0f40023061b9</span><br><span class="line"> else</span><br><span class="line"> search --no-floppy --fs-uuid --set=root cdbb3f80-2c48-4c5c-bc72-0f40023061b9</span><br><span class="line"> fi</span><br><span class="line"> linux16 /vmlinuz-3.10.0-957.5.1.el7.x86_64 root=/dev/mapper/cl-root ro crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap ifname=enp0s3:08:00:27:37:8e:ba rhgb elevator=deadline systemd.unit=graphical.target</span><br></pre></td></tr></table></figure><p>paste menuentry in 40_custom, then change name to “my centOS” and config it to fit your requirement, creating a new menu.<br>Don’t forget mkconfig, after doing that, the new menu can be created.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">grub2-mkconfig -o /boot/grub2/grub.cfg</span><br></pre></td></tr></table></figure><p>The new menu will be created at the bottom of /boot/grub.d/grub.cfg, when booting linux, it will show on the menu.</p><a id="more"></a><h3 id="Change-grub-timeout"><a href="#Change-grub-timeout" class="headerlink" title="Change grub timeout"></a>Change grub timeout</h3><p>Sometimes, the default timeout is too long, users can change it.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">cd /etc/default/grub</span><br></pre></td></tr></table></figure><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">GRUB_TIMEOUT=25</span><br><span class="line">GRUB_TIMEOUT_STYLE=menu</span><br><span class="line">GRUB_DISTRIBUTOR="$(sed 's, release .*$,,g' /etc/system-release)"</span><br><span class="line">GRUB_DEFAULT=0</span><br><span class="line">GRUB_DISABLE_SUBMENU=true</span><br><span class="line">GRUB_TERMINAL_OUTPUT="console"</span><br><span class="line">GRUB_CMDLINE_LINUX="crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap ifname=enp0s3:08:00:27:37:8e:ba elevator=deadline elevator=deadline"</span><br><span class="line">GRUB_DISABLE_RECOVERY="true"</span><br></pre></td></tr></table></figure><p>change to what time you want</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">GRUB_TIMEOUT=5</span><br></pre></td></tr></table></figure><h3 id="Showing-process-during-booting"><a href="#Showing-process-during-booting" class="headerlink" title="Showing process during booting"></a>Showing process during booting</h3><p>If you want to see the process detail during boot, delete the “rhgb quiet”</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">linux16 /vmlinuz-3.10.0-957.5.1.el7.x86_64 root=/dev/mapper/cl-root ro crashkernel=auto rd.lvm.lv=cl/root rd.lvm.lv=cl/swap ifname=enp0s3:08:00:27:37:8e:ba rhgb elevator=deadline systemd.unit=graphical.target</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Linux: backup</title>
<link href="/2018/01/31/Linux-backup/"/>
<url>/2018/01/31/Linux-backup/</url>
<content type="html"><![CDATA[<h4 id="Important-directory-needed-to-backup"><a href="#Important-directory-needed-to-backup" class="headerlink" title="Important directory needed to backup"></a>Important directory needed to backup</h4><p>/etc<br>/home<br>/var/spool/mail<br>/var/spool/{at,cron}<br>/boot<br>/root<br>/usr/local<br>/var/www<br>/srv/www<br>/var/lib/mysql for apache server</p><h4 id="backup-methods"><a href="#backup-methods" class="headerlink" title="backup methods"></a>backup methods</h4><p>dd :</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">dd if=/dev/sda of=/dev/sdb</span><br></pre></td></tr></table></figure><p>let /dev/sdb keep all the backup for /dev/sda</p><p>cpio:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">find / -print |cpio -covB > dev/st0</span><br></pre></td></tr></table></figure><p>backup all / to the tape</p><a id="more"></a><p>xfsdump:<br>Fully backup:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">xfsdump -l 0 -L 'home_full' -M 'home_full' -f /backupdata/home.dump /home</span><br></pre></td></tr></table></figure><p>Incrementally backup:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">xfsdump -l 1 -L 'full-1' -M 'full-1' -f /backupdata/home.dump1 /home</span><br></pre></td></tr></table></figure><p>it means backup /home to /backupdata<br>tar:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">tar --exclude /proc --exclude /mnt --exclude /tmp --exclude /backupdata -jcvp -f /backupdata/system.tar.bz2 /</span><br></pre></td></tr></table></figure><p>f: filename<br>v:verbose<br>c:create<br>p:preserve-permissions<br>Add date to backup file :</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">tar -jcvpf filename.`date +%Y-%m-%d`.tar.bz2 /var/lib/mysql</span><br></pre></td></tr></table></figure><p>rsync:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">rsync -av /home /backupdata/</span><br></pre></td></tr></table></figure><p>Backup /home to /backupdata</p>]]></content>
<tags>
<tag> linux </tag>
</tags>
</entry>
<entry>
<title>Making charts with python matplotlib</title>
<link href="/2018/01/08/Making-charts-with-python-matplotlib/"/>
<url>/2018/01/08/Making-charts-with-python-matplotlib/</url>
<content type="html"><![CDATA[<h2 id="About-matplotlib"><a href="#About-matplotlib" class="headerlink" title="About matplotlib"></a>About matplotlib</h2><p><a href="https://matplotlib.org/" target="_blank" rel="noopener">Matplotlib</a> is a python plotting library which can help generate plots, and charts. </p><h2 id="Installation"><a href="#Installation" class="headerlink" title="Installation"></a>Installation</h2><p>Install python3 matplotlib on my Ubuntu 16.04 box</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt install python3-matplotlib</span><br></pre></td></tr></table></figure><h2 id="Creating-a-simple-chart-in-python-shell"><a href="#Creating-a-simple-chart-in-python-shell" class="headerlink" title="Creating a simple chart in python shell"></a>Creating a simple chart in python shell</h2><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">$ python3</span><br><span class="line">Python <span class="number">3.5</span><span class="number">.2</span> (default, Nov <span class="number">23</span> <span class="number">2017</span>, <span class="number">16</span>:<span class="number">37</span>:<span class="number">01</span>) </span><br><span class="line">[GCC <span class="number">5.4</span><span class="number">.0</span> <span class="number">20160609</span>] on linux</span><br><span class="line">Type <span class="string">"help"</span>, <span class="string">"copyright"</span>, <span class="string">"credits"</span> <span class="keyword">or</span> <span class="string">"license"</span> <span class="keyword">for</span> more information.</span><br><span class="line"><span class="meta">>>> </span><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="meta">>>> </span>plt</span><br><span class="line"><module <span class="string">'matplotlib.pyplot'</span> <span class="keyword">from</span> <span class="string">'/usr/lib/python3/dist-packages/matplotlib/pyplot.py'</span>></span><br><span class="line"><span class="meta">>>> </span>x = range(<span class="number">10</span>) <span class="comment"># create X coordinate data</span></span><br><span class="line"><span class="meta">>>> </span>y = [ i*i <span class="keyword">for</span> i <span class="keyword">in</span> x ] <span class="comment"># create Y coordinate data with square of data on X.</span></span><br><span class="line"><span class="meta">>>> </span>plt.plot(x, y)</span><br><span class="line">[<matplotlib.lines.Line2D object at <span class="number">0x7f4f7d4433c8</span>>]</span><br><span class="line"><span class="meta">>>> </span>plt.show()</span><br></pre></td></tr></table></figure><p>Then I got the chart</p><img src="/2018/01/08/Making-charts-with-python-matplotlib/plot_example.png" class=""><a id="more"></a><h2 id="Creating-a-bar-chart"><a href="#Creating-a-bar-chart" class="headerlink" title="Creating a bar chart"></a>Creating a bar chart</h2><p>Next I wrote a script to generate a bar chart of the <a href="https://www.currentresults.com/Weather/California/Places/san-francisco-temperatures-by-month-average.php" target="_blank" rel="noopener">average monthly high temperature of San Francisco</a>.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line"> print(<span class="string">'This program will generate a bar chart of the monthly average high temperature of San Francisco'</span>)</span><br><span class="line"> <span class="comment"># Create the list of X and Y cordinate data</span></span><br><span class="line"> left_edges = [ i <span class="keyword">for</span> i <span class="keyword">in</span> range(<span class="number">12</span>)]</span><br><span class="line"> temperature = [<span class="number">57</span>, <span class="number">60</span>, <span class="number">62</span>, <span class="number">63</span>, <span class="number">64</span>, <span class="number">66</span>, <span class="number">67</span>, <span class="number">68</span>, <span class="number">70</span>, <span class="number">69</span>, <span class="number">63</span>, <span class="number">57</span>]</span><br><span class="line"> bar_width = <span class="number">0.3</span></span><br><span class="line"> plt.bar(left_edges, temperature, bar_width, color=(<span class="string">'r'</span>,<span class="string">'g'</span>,<span class="string">'b'</span>,<span class="string">'r'</span>,<span class="string">'k'</span>,<span class="string">'r'</span>,<span class="string">'g'</span>,<span class="string">'b'</span>,<span class="string">'r'</span>,<span class="string">'k'</span>,<span class="string">'r'</span>,<span class="string">'g'</span>))</span><br><span class="line"> <span class="comment"># Add a title</span></span><br><span class="line"> plt.title(<span class="string">'San Francisco Climate Graph-Chart'</span>)</span><br><span class="line"> <span class="comment"># Add the label to the axes</span></span><br><span class="line"> plt.xlabel(<span class="string">'Month'</span>)</span><br><span class="line"> plt.ylabel(<span class="string">'Average High Temperature'</span>)</span><br><span class="line"> <span class="comment"># Customize the tick marks</span></span><br><span class="line"> months = [<span class="string">'Jan'</span>, <span class="string">'Feb'</span>, <span class="string">'Mar'</span>, <span class="string">'Apr'</span>, <span class="string">'May'</span>, <span class="string">'Jun'</span>, <span class="string">'Jul'</span>, <span class="string">'Aug'</span>, <span class="string">'Sep'</span>, <span class="string">'Oct'</span>, <span class="string">'Nov'</span>, <span class="string">'Dec'</span>]</span><br><span class="line"> plt.xticks(left_edges, months)</span><br><span class="line"> plt.yticks([<span class="number">50</span>, <span class="number">55</span>, <span class="number">60</span>, <span class="number">65</span>, <span class="number">70</span>, <span class="number">75</span>], [<span class="string">'50F'</span>, <span class="string">'55F'</span>, <span class="string">'60F'</span>, <span class="string">'65F'</span>, <span class="string">'70F'</span>, <span class="string">'75F'</span>])</span><br><span class="line"> <span class="comment"># Add a grid</span></span><br><span class="line"> plt.grid(<span class="literal">True</span>)</span><br><span class="line"> <span class="comment"># Display the bar chart graph</span></span><br><span class="line"> plt.show()</span><br><span class="line"></span><br><span class="line">main()</span><br></pre></td></tr></table></figure><p>Then I got the bar chart</p><img src="/2018/01/08/Making-charts-with-python-matplotlib/bar_chart.png" class=""><h2 id="Creating-a-pie-chart"><a href="#Creating-a-pie-chart" class="headerlink" title="Creating a pie chart"></a>Creating a pie chart</h2><p>This script I wrote generates a pie chart of the <a href="https://suburbanstats.org/population/how-many-people-live-in-california" target="_blank" rel="noopener">Population for California by race in 2017 and 2016</a></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> matplotlib.pyplot <span class="keyword">as</span> plt</span><br><span class="line"><span class="function"><span class="keyword">def</span> <span class="title">main</span><span class="params">()</span>:</span></span><br><span class="line"> print(<span class="string">'This program will generate a pie chart of Population for California by race in 2017 and 2016'</span>)</span><br><span class="line"> values = [<span class="number">21454</span>, <span class="number">14014</span>, <span class="number">2299</span>, <span class="number">4861</span>, <span class="number">363</span>, <span class="number">8600</span>]</span><br><span class="line"> plt.pie(values)</span><br><span class="line"> slice_label = [<span class="string">'White'</span>, <span class="string">'Hispanic'</span>, <span class="string">'Black'</span>, <span class="string">'Asian'</span>, <span class="string">'American Indian'</span>, <span class="string">'Others'</span>]</span><br><span class="line"> <span class="comment"># Add a title</span></span><br><span class="line"> plt.title(<span class="string">'Population for California by race in 2017 and 2016'</span>)</span><br><span class="line"> <span class="comment"># Add the label to the axes</span></span><br><span class="line"> plt.pie(values, labels=slice_label)</span><br><span class="line"> <span class="comment"># save the chart image</span></span><br><span class="line"> plt.savefig(<span class="string">'pie_chart.png'</span>)</span><br><span class="line"> <span class="comment"># Display the pie chart graph</span></span><br><span class="line"> plt.show()</span><br><span class="line">main()</span><br></pre></td></tr></table></figure><p>The pie chart I got</p><img src="/2018/01/08/Making-charts-with-python-matplotlib/pie_chart.png" class=""><h2 id="Save-the-chart-to-a-image-file"><a href="#Save-the-chart-to-a-image-file" class="headerlink" title="Save the chart to a image file"></a>Save the chart to a image file</h2><p>Just use the savefig() method.</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">plt.savefig(<span class="string">'bar_chart.png'</span>)</span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> python </tag>
</tags>
</entry>
<entry>
<title>Decorating my blog with Hexo - theme, features, tips</title>
<link href="/2017/12/21/Decorating-my-blog-with-Hexo-theme-and-more-featurs/"/>
<url>/2017/12/21/Decorating-my-blog-with-Hexo-theme-and-more-featurs/</url>
<content type="html"><![CDATA[<h3 id="Change-hexo-blog-theme-NexT"><a href="#Change-hexo-blog-theme-NexT" class="headerlink" title="Change hexo blog theme - NexT"></a>Change hexo blog theme - NexT</h3><p>Change to hexo blog theme <a href="http://theme-next.iissnan.com/" target="_blank" rel="noopener">NexT</a> which has neat look and supports a lot cool features.<br>Download the theme</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ git <span class="built_in">clone</span> https://github.com/iissnan/hexo-theme-next themes/next</span><br></pre></td></tr></table></figure><p>Edit blog configuration file _config.yml, change to theme from the default langscape to next</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">theme: next</span><br></pre></td></tr></table></figure><p>Edit theme configuration file themes/next/_config.yml, change list scheme from default Muse to Pisces, see more about <a href="https://github.com/iissnan/hexo-theme-next" target="_blank" rel="noopener">NexT theme scheme</a></p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"># Schemes</span><br><span class="line">#scheme: Muse</span><br><span class="line">#scheme: Mist</span><br><span class="line">scheme: Pisces</span><br></pre></td></tr></table></figure><p>I prefer the scheme Pisces because it has menu and links and other stuff showing on left side of the blog content.</p><a id="more"></a><h3 id="Add-blog-owner-avatar"><a href="#Add-blog-owner-avatar" class="headerlink" title="Add blog owner avatar"></a>Add blog owner avatar</h3><p>I’d like my avatar showing up in my blog, and I have avatar uploaded to my github account.<br>Locate the avatar image link of my github account, mine is <a href="https://avatars0.githubusercontent.com/u/34552097" target="_blank" rel="noopener">https://avatars0.githubusercontent.com/u/34552097</a></p><p>Edit blog configuration file _config.yml, add a line in # Site section</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">avatar: https://avatars0.githubusercontent.com/u/34552097</span><br></pre></td></tr></table></figure><h3 id="Create-categories-page-tags-page-and-about-page"><a href="#Create-categories-page-tags-page-and-about-page" class="headerlink" title="Create categories page, tags page, and about page"></a>Create categories page, tags page, and about page</h3><h4 id="create-categories-page"><a href="#create-categories-page" class="headerlink" title="create categories page"></a>create categories page</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new page "categories"</span><br><span class="line">INFO Created: ~/vmnet8.github.io/source/categories/index.md</span><br></pre></td></tr></table></figure><p>Edit source/categories/index.md, set page type to “categories”, change title to “Categories”</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">title: Categories</span><br><span class="line">date: 2017-12-17 22:39:34</span><br><span class="line">type: categories</span><br></pre></td></tr></table></figure><h4 id="create-tags-page"><a href="#create-tags-page" class="headerlink" title="create tags page"></a>create tags page</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new page "tags"</span><br><span class="line">INFO Created: ~/vmnet8.github.io/source/tags/index.md</span><br></pre></td></tr></table></figure><p>Edit source/tags/index.md, set type to “tags”, change title to “Tags”</p><h4 id="create-about-page"><a href="#create-about-page" class="headerlink" title="create about page"></a>create about page</h4><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new page "about"</span><br><span class="line">INFO Created: ~/vmnet8.github.io/source/about/index.md</span><br></pre></td></tr></table></figure><p>Edit source/about/index.md, change title to “About me”, and add description of about myself.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">---</span><br><span class="line">title: About Me</span><br><span class="line">date: 2017-12-18 11:41:02</span><br><span class="line">---</span><br><span class="line">My name is Anna X.</span><br></pre></td></tr></table></figure><p>Edit theme configuration file themes/next/_config.yml, enable categories, tags, and about menu.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">menu:</span><br><span class="line"> home: / || home</span><br><span class="line"> about: /about/ || user</span><br><span class="line"> tags: /tags/ || tags</span><br><span class="line"> categories: /categories/ || th</span><br><span class="line"> archives: /archives/ || archive</span><br></pre></td></tr></table></figure><p>Edit the post source/_post/1st-blog.md, add categories and tags</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">title: 1st blog</span><br><span class="line">date: 2017-12-17 21:36:48</span><br><span class="line">tags:</span><br><span class="line"> - blog</span><br><span class="line"> - hexo</span><br><span class="line">categories:</span><br><span class="line"> - life</span><br></pre></td></tr></table></figure><h3 id="Enable-social-links"><a href="#Enable-social-links" class="headerlink" title="Enable social links"></a>Enable social links</h3><p>Edit theme configuration file, themes/next/_config.yml, enable social links, change icons_only from false to true.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">social:</span><br><span class="line"> GitHub: https://github.com/vmnet8 || github</span><br><span class="line"> E-Mail: mailto:[email protected] || envelope</span><br><span class="line"> #Google: https://plus.google.com/yourname || google</span><br><span class="line"> #Twitter: https://twitter.com/username || twitter</span><br><span class="line"> #FB Page: https://www.facebook.com/yourname || facebook</span><br><span class="line"> #VK Group: https://vk.com/yourname || vk</span><br><span class="line"> #StackOverflow: https://stackoverflow.com/yourname || stack-overflow</span><br><span class="line"> #YouTube: https://youtube.com/yourname || youtube</span><br><span class="line"> #Instagram: https://instagram.com/yourname || instagram</span><br><span class="line"> #Skype: skype:yourname?call|chat || skype</span><br><span class="line"> Linkedin: https://www.linkedin.com/in/anna_x/ || linkedin</span><br><span class="line"></span><br><span class="line">social_icons:</span><br><span class="line"> enable: true</span><br><span class="line"> icons_only: true</span><br><span class="line"> transition: false</span><br></pre></td></tr></table></figure><h3 id="Add-RSS"><a href="#Add-RSS" class="headerlink" title="Add RSS"></a>Add RSS</h3><p>Install hexo-generator-feed plugin</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ npm install hexo-generator-feed --save</span><br></pre></td></tr></table></figure><p>Edit blog configuration file _config.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">type: atom</span><br><span class="line">path: atom.xml</span><br><span class="line">limit: 20</span><br><span class="line">hub:</span><br><span class="line">content:</span><br><span class="line">content_limit: 140</span><br><span class="line">content_limit_delim: ' '</span><br></pre></td></tr></table></figure><h3 id="Adding-image-to-blog-post"><a href="#Adding-image-to-blog-post" class="headerlink" title="Adding image to blog post"></a>Adding image to blog post</h3><p>I’d like the image for a blog post to be in a blog post specific folder, so I enabled the <a href="https://hexo.io/docs/asset-folders.html" target="_blank" rel="noopener">“post_asset_folder”</a><br>Edit blog configuration file _config.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">post_asset_folder: true</span><br></pre></td></tr></table></figure><p>When I do “hexo new ‘My 1st blog’”, a blog specific asset folder named “My-1st-blog” will be created under source/_post along with the post markdown file My-1st-blog.md .</p><p>Copy the image file ‘penguins.png’ to source/_post/My-1st-blog<br>Edit source_post/My-1st-blog.md, reference the image in the markdown file like this</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">This is my first blog post.</span><br><span class="line">{% asset_img example.png This is an example image %}</span><br></pre></td></tr></table></figure><h3 id="Enable-post-comment-with-disqus"><a href="#Enable-post-comment-with-disqus" class="headerlink" title="Enable post comment with disqus"></a>Enable post comment with disqus</h3><p>Sign up <a href="https://disqus.com/" target="_blank" rel="noopener">https://disqus.com/</a>, you can sign up with twitter, facebook, or google account. On next step, choose “I want to install Disqus on my site”, mark the shortname of the site. My shortname is annax</p><p>NexT theme already supports disqus, edit theme configuration theme/next/_config.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">disqus:</span><br><span class="line"> enable: true</span><br><span class="line"> shortname: annax</span><br><span class="line"> count: true</span><br></pre></td></tr></table></figure><p>Then each blog post, and about page, and tags page will have adding comment embeded below the content.<br>If you want to disable comment for specific page or post, add “comments: false” to post/page front-matter<br>For me, I disable comment for tags page.</p><h3 id="Page-and-site-view-count"><a href="#Page-and-site-view-count" class="headerlink" title="Page and site view count"></a>Page and site view count</h3><p>Enable busuanzi_count in theme next configuration file, edit theme/next/_config.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">busuanzi_count:</span><br><span class="line"> # count values only if the other configs are false</span><br><span class="line"> enable: true</span><br></pre></td></tr></table></figure><p>Then the blog home page will have site view count at the bottom and each post page will have post view count at the bottom.</p><h3 id="Enable-local-search"><a href="#Enable-local-search" class="headerlink" title="Enable local search"></a>Enable local search</h3><p>Install hexo-generator-searchdb plugin</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ npm install hexo-generator-searchdb --save</span><br></pre></td></tr></table></figure><p>Edit blog configuration file _config.yml, add</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">search:</span><br><span class="line"> path: search.xml</span><br><span class="line"> field: post</span><br><span class="line"> format: html</span><br><span class="line"> limit: 10000</span><br></pre></td></tr></table></figure><p>Edit theme configuration file theme/next/_config.yml</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">local_search:</span><br><span class="line"> enable: true</span><br></pre></td></tr></table></figure><p>Then the blog menu will have a “search” button</p><h3 id="Tips"><a href="#Tips" class="headerlink" title="Tips"></a>Tips</h3><p>If the blog is too long and you don’t want the whole blog content show in the blog home page, add a line to blog post markdown file where you want to insert a “Read more” button.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><!--more--></span><br></pre></td></tr></table></figure>]]></content>
<tags>
<tag> hexo </tag>
</tags>
</entry>
<entry>
<title>First blog - Setting up my github pages blog with Hexo</title>
<link href="/2017/12/21/First-blog-Setting-up-my-github-pages-blog-with-Hexo/"/>
<url>/2017/12/21/First-blog-Setting-up-my-github-pages-blog-with-Hexo/</url>
<content type="html"><![CDATA[<h3 id="Introduction"><a href="#Introduction" class="headerlink" title="Introduction"></a>Introduction</h3><h4 id="Github-pages"><a href="#Github-pages" class="headerlink" title="Github pages"></a>Github pages</h4><p><a href="https://pages.github.com/" target="_blank" rel="noopener">Github pages</a> is a github service that can host personal or project web pages directly from a github repository. This is ideal place to host my blog site because it is fast, and I will have a <username>github.io domain, and there are some really cool blog frameworks that support deploying blog site to Github Pages.</p><h4 id="Hexo"><a href="#Hexo" class="headerlink" title="Hexo"></a>Hexo</h4><p>The most popular blog framework that supports github pages is Jekyll It seems, because it is officially recommended by github. But I found another framework <a href="https://hexo.io/" target="_blank" rel="noopener">Hexo</a>, which compared to Jekyll, is similar but much easier to use, I only need to know 4 commands,</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">hexo n # new post</span><br><span class="line">hexo g # generate</span><br><span class="line">hexo s # server</span><br><span class="line">hexo d # deploy</span><br></pre></td></tr></table></figure><p>After I checked out some Hexo built blog sites, I decied to go with Hexo. Hexo is wirtten in node.js, and it supports a lot powerfull plugins and has cool themes.</p><a id="more"></a><h3 id="Install-Hexo"><a href="#Install-Hexo" class="headerlink" title="Install Hexo"></a>Install Hexo</h3><p>Follow <a href="https://hexo.io/docs/index.html" target="_blank" rel="noopener">Hexo doc</a></p><h4 id="Set-up-Github-Pages"><a href="#Set-up-Github-Pages" class="headerlink" title="Set up Github Pages"></a>Set up Github Pages</h4><p>Create github pages repository on github.com, the repository name is vmnet8.github.io where vmnet8 is my github account.</p><h4 id="Install-and-configure-Git-which-Hexo-requires"><a href="#Install-and-configure-Git-which-Hexo-requires" class="headerlink" title="Install and configure Git which Hexo requires"></a>Install and configure Git which Hexo requires</h4><p>Install git. I am coding and will be blogging on my Ubuntu Linux, to use Hexo to deploy my blog site to github pages, I need to install and configure git first. Install:</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt install git</span><br></pre></td></tr></table></figure><p>Configure git with github</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ git config --global user.name "Anna X"</span><br><span class="line">$ git config --global user.email "[email protected]"</span><br><span class="line">$ git config --list</span><br></pre></td></tr></table></figure><p>Add my <a href="https://github.com/settings/keys" target="_blank" rel="noopener">ssh public key to github</a>. I will have Hexo deploy to github pages over ssh without asking username and password.</p><h4 id="Install-Node-js"><a href="#Install-Node-js" class="headerlink" title="Install Node.js"></a>Install Node.js</h4><p>Follow <a href="https://nodejs.org/en/download/package-manager/#debian-and-ubuntu-based-linux-distributions" target="_blank" rel="noopener">node.js doc</a><br>Install node.js using package manager - APT</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -</span><br><span class="line">$ sudo apt install -y nodejs</span><br></pre></td></tr></table></figure><p>As of 12/21/2017, node.js version is v8.9.3</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ node -v</span><br><span class="line">v8.9.3</span><br></pre></td></tr></table></figure><h4 id="Install-hexo-cli"><a href="#Install-hexo-cli" class="headerlink" title="Install hexo-cli"></a>Install hexo-cli</h4><p>Follow <a href="https://hexo.io/docs/index.html" target="_blank" rel="noopener">Hexo doc</a></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">sudo npm install -g hexo-cli</span><br></pre></td></tr></table></figure><p>As of 12/21/2017, hexo version 3.4.3</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">$ hexo version</span><br><span class="line">hexo: 3.4.3</span><br><span class="line">hexo-cli: 1.0.4</span><br><span class="line">os: Linux 4.4.0-103-generic linux x64</span><br><span class="line">http_parser: 2.7.0</span><br><span class="line">node: 8.9.3</span><br><span class="line">v8: 6.1.534.48</span><br><span class="line">uv: 1.15.0</span><br><span class="line">zlib: 1.2.11</span><br><span class="line">ares: 1.10.1-DEV</span><br><span class="line">modules: 57</span><br><span class="line">nghttp2: 1.25.0</span><br><span class="line">openssl: 1.0.2n</span><br><span class="line">icu: 59.1</span><br><span class="line">unicode: 9.0</span><br><span class="line">cldr: 31.0.1</span><br><span class="line">tz: 2017b</span><br></pre></td></tr></table></figure><h3 id="Build-my-blog-using-Hexo"><a href="#Build-my-blog-using-Hexo" class="headerlink" title="Build my blog using Hexo"></a>Build my blog using Hexo</h3><h4 id="Initialize-my-blog-folder"><a href="#Initialize-my-blog-folder" class="headerlink" title="Initialize my blog folder"></a>Initialize my blog folder</h4><p>Follow <a href="https://hexo.io/docs/setup.html" target="_blank" rel="noopener">Hexo setup doc</a>, I use “vmnet8.github.io” as my blog folder.</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ hexo init vmnet8.github.io</span><br><span class="line">$ <span class="built_in">cd</span> vmnet.github.io/</span><br><span class="line">$ npm install</span><br></pre></td></tr></table></figure><h4 id="Basic-blog-site-configuration"><a href="#Basic-blog-site-configuration" class="headerlink" title="Basic blog site configuration"></a>Basic blog site configuration</h4><p>Edit blog configuration file _config.yml in blog folder.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line"># Site</span><br><span class="line">title: Anna's blog</span><br><span class="line">subtitle: My tech blog</span><br><span class="line">description:</span><br><span class="line">author: Anna X</span><br><span class="line">language: en</span><br><span class="line">timezone: America/Los_Angeles</span><br></pre></td></tr></table></figure><p>Note that when editing _config.yml file which is <a href="http://www.yaml.org/start.html" target="_blank" rel="noopener">YAML format file</a>, for key-value pairs format like “key: value”, there should be a space after “:”, same for sequence item, there should be a space after “-“</p><h3 id="Create-blog-post-using-Hexo"><a href="#Create-blog-post-using-Hexo" class="headerlink" title="Create blog post using Hexo"></a>Create blog post using Hexo</h3><h4 id="Create-my-first-blog-post"><a href="#Create-my-first-blog-post" class="headerlink" title="Create my first blog post"></a>Create my first blog post</h4><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new <span class="string">"1st blog"</span></span><br><span class="line">INFO Created: ~/vmnet8.github.io/<span class="built_in">source</span>/_posts/1st-blog.md</span><br></pre></td></tr></table></figure><p> Edit the blog post file 1st-blog.md</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ vi <span class="built_in">source</span>/_post/1st-blog.md</span><br></pre></td></tr></table></figure><p>The blog post .md file is <a href="https://en.wikipedia.org/wiki/Markdown" target="_blank" rel="noopener">markdown format</a> file with a header called “Front matter”</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">---</span><br><span class="line">title: 1st blog</span><br><span class="line">date: 2017-12-21 12:44:07</span><br><span class="line">---</span><br><span class="line">This is my first blog post.</span><br></pre></td></tr></table></figure><p>The front matter is <a href="http://www.yaml.org/start.html" target="_blank" rel="noopener">YAML format</a> .<br>The blog content below front matter is <a href="https://en.wikipedia.org/wiki/Markdown" target="_blank" rel="noopener">markdown format</a> which is a lightweight markup language with plain text formatting syntax. It is designed so that it can be converted to HTML.</p><p>When creating blog folder using “hexo init”, a sample blog post source/_posts/hello-world.md will be created automatically, you can cat it for reference.</p><h4 id="Test-blog-post"><a href="#Test-blog-post" class="headerlink" title="Test blog post"></a>Test blog post</h4><p>Run “hexo g” to generate the static html files and “hexo server” to start http service on port 4000 for blog access.</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ hexo g</span><br><span class="line">$ hexo s</span><br><span class="line">INFO Start processing</span><br><span class="line">INFO Hexo is running at http://localhost:4000/. Press Ctrl+C to stop.</span><br></pre></td></tr></table></figure><p>Then go to <a href="http://localhost:4000/" target="_blank" rel="noopener">http://localhost:4000/</a> in browser.<br>Notes:</p><ul><li>Every time you run hexo command, you current dir should be the hexo blog folder.</li><li>Note: every time you make change to the configuration file or any page or post, you need to do “hexo g” to generate the new static html files. If something wrong happens due to wrong syntax in yml configuration file or md markdown file, you may need to do “hexo clean” and then “hexo g” to re-generate all blog html files.</li></ul><h3 id="Deploy-blog-to-github-pages"><a href="#Deploy-blog-to-github-pages" class="headerlink" title="Deploy blog to github pages"></a>Deploy blog to github pages</h3><p>Follow <a href="https://hexo.io/docs/deployment.html" target="_blank" rel="noopener">Hexo deployment doc</a><br>Install git hexo deployer plugin.</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">npm install hexo-deployer-git --save</span><br></pre></td></tr></table></figure><p>Note: When you install hexo module by “npm install <module> –save”, your current dir should also be the blog fold.<br> Configure deployment to higthub, edit blog configuration file _config.yml in blog folder</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">deploy:</span><br><span class="line"> type: git</span><br><span class="line"> repo: [email protected]:vmnet8/vmnet8.github.io.git</span><br><span class="line"> branch: master</span><br></pre></td></tr></table></figure><p>I use ssh instead of https for my github pages repository access for Hexo, so I won’t need to enter username/password for deployment to github.<br> Deploy to github pages</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo d</span><br></pre></td></tr></table></figure><p>Then go to my github pages <a href="https://vmnet8.github.io">https://vmnet8.github.io</a> in browser.</p>]]></content>
<tags>
<tag> hexo </tag>
</tags>
</entry>
<entry>
<title>Hello World</title>
<link href="/2017/12/19/hello-world/"/>
<url>/2017/12/19/hello-world/</url>
<content type="html"><![CDATA[<p>Welcome to <a href="https://hexo.io/" target="_blank" rel="noopener">Hexo</a>! This is your very first post. Check <a href="https://hexo.io/docs/" target="_blank" rel="noopener">documentation</a> for more info. If you get any problems when using Hexo, you can find the answer in <a href="https://hexo.io/docs/troubleshooting.html" target="_blank" rel="noopener">troubleshooting</a> or you can ask me on <a href="https://github.com/hexojs/hexo/issues" target="_blank" rel="noopener">GitHub</a>.</p><h2 id="Quick-Start"><a href="#Quick-Start" class="headerlink" title="Quick Start"></a>Quick Start</h2><h3 id="Create-a-new-post"><a href="#Create-a-new-post" class="headerlink" title="Create a new post"></a>Create a new post</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo new <span class="string">"My New Post"</span></span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/writing.html" target="_blank" rel="noopener">Writing</a></p><h3 id="Run-server"><a href="#Run-server" class="headerlink" title="Run server"></a>Run server</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo server</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/server.html" target="_blank" rel="noopener">Server</a></p><h3 id="Generate-static-files"><a href="#Generate-static-files" class="headerlink" title="Generate static files"></a>Generate static files</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo generate</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/generating.html" target="_blank" rel="noopener">Generating</a></p><h3 id="Deploy-to-remote-sites"><a href="#Deploy-to-remote-sites" class="headerlink" title="Deploy to remote sites"></a>Deploy to remote sites</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ hexo deploy</span><br></pre></td></tr></table></figure><p>More info: <a href="https://hexo.io/docs/deployment.html" target="_blank" rel="noopener">Deployment</a></p>]]></content>
<tags>
<tag> blog </tag>