forked from FINkit/learn-bosh-on-gcp
-
Notifications
You must be signed in to change notification settings - Fork 0
/
index.html
665 lines (529 loc) · 25.5 KB
/
index.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
<!DOCTYPE html>
<html>
<head>
<script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script type="text/javascript" src="fullpage_js/vendors/jquery.slimscroll.min.js"></script>
<script type="text/javascript" src="fullpage_js/jquery.fullPage.min.js"></script>
<link rel="stylesheet" type="text/css" href="fullpage_js/jquery.fullPage.css" />
<link rel="stylesheet" type="text/css" href="css/style.css" />
<script type="text/javascript">
$(document).ready(function() {
$('#fullpage').fullpage({
navigation: true,
navigationPosition: 'right',
scrollOverflow: true,
css3: false,
anchors: ["introduction", "prepare", "create_bosh_bastion", "create_bosh_director", "log_in", "deploy", "create_release", "upload_stemcell", "update_cloud_config", "run_deploy", "modify", "modify_release", "scale", "change_properties", "when_something_goes_wrong", "failing_service", "failing_vm", "failing_deploy", "done"],
navigationTooltips: ["Introduction", "Prepare", "Create BOSH Bastion", "Create BOSH Director", "Log in", "Deploy", "Create release", "Upload stemcell", "Update cloud config", "Run deploy", "Modify deployment", "Modify release", "Scale", "Change properties", "When something goes wrong", "Failing service", "Failing VM", "Failing deploy", "Done!"],
});
});
</script>
<title>A Guide to Using BOSH</title>
</head>
<body>
<div id="fullpage">
<div class="section intro"><div class="center-column"><h2>A Guide to Using</h2>
<h1>BOSH on GCP</h1>
<p>This tutorial aims to give a <i>relatively</i> simple introduction to using BOSH on Google Cloud Platform (GCP).</p>
<p>It is <b>heavily</b> based on the excellent <a href="http://mariash.github.io/learn-bosh/">A Guide to Using BOSH</a> tutorial, written by <a href="https://github.com/mariash">Maria Shaldibina</a>.</p>
<p>Prerequisites:
<ul>
<li>A GCP account (at the time of writing a <a href="https://cloud.google.com/free/">Free Tier</a> is available)</li>
</ul>
</p>
<p>Learn more:
<ul>
<li><a href="https://cloud.google.com/getting-started/">cloud.google.com: Getting Started with Google Cloud Platform</a></li>
<li><a href="http://bosh.io/docs/about.html">bosh.io: What is BOSH?</a></li>
<li><a href="http://bosh.io/docs/problems.html">bosh.io: What Problems Does BOSH Solve?</a></li>
</ul>
</p>
<button onClick="$.fn.fullpage.moveSectionDown();" id="start-button">Start</button>
</div></div>
<div class="section prepare"><div class="center-column"><h1>Prepare</h1>
<p>We are going to use the BOSH Environment created using the scripts in the <a href="https://github.com/finkit/bosh-on-gcp">bosh-on-gcp</a> repository. First we will create a BOSH Bastion in a dedicated GCP Project, the BOSH Bastion will have the BOSH CLI v2 installed. It is from this BOSH Bastion that we will create a BOSH Director. Once the BOSH Director is running we will use the BOSH CLI on the BOSH Bastion to send commands to the BOSH Director.</p>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/bosh-components.html">bosh.io: Components</a></li>
</ul>
</p>
<button onClick="$.fn.fullpage.moveSectionDown();" id="next-button">Next</button>
</div></div>
<div class="section prepare"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>1</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Create BOSH Bastion</h2>
</div>
</div>
<p>The BOSH Bastion is the <i>jumpbox</i> we will use to access the BOSH Director, and the BOSH Environment in general. It is created using the Google Cloud SDK and <a href="https://www.terraform.io/">Terraform</a>. To install the Google Cloud SDK follow the instructions on the <a href="https://cloud.google.com/sdk/">Cloud SDK</a> page.</p>
<p>Once the Cloud SDK is installed, initialize a gcloud session:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ gcloud init</h4>
</div>
<p>To create a GCP Project and BOSH Bastion, clone the <a href="https://github.com/finkit/bosh-on-gcp">bosh-on-gcp</a> repository and execute the <code>create-project.sh</code> <code>build-bosh-resources.sh</code> scripts:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ git clone https://github.com/finkit/bosh-on-gcp</h4>
<h4 class="terminal-code-text">$ cd bosh-on-gcp</h4>
<h4 class="terminal-code-text">$ ./create-project.sh -p <GCP_PROJECT></h4>
<h4 class="terminal-code-text">$ cd terraform</h4>
<h4 class="terminal-code-text">$ ./build-bosh-resources.sh -p <GCP_PROJECT></h4>
</div>
<p>Learn more:
<ul>
<li><a href="https://cloud.google.com/sdk/docs/">cloud.google.com: Cloud SDK</a></li>
<li><a href="https://www.terraform.io/docs/index.html">terraform.io: Terraform Documentation</a></li>
</ul>
</p>
</div></div>
<div class="section prepare"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>2</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Create BOSH Director</h2>
</div>
</div>
<p>To create the BOSH Director, ssh into the BOSH Bastion VM and execute the <code>create-bosh-director.sh</code> script:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ gcloud compute ssh bosh-bastion</h4>
<h4 class="terminal-code-text">$ . ./create-bosh-director.sh</h4>
</div>
<p>Note the extra leading '.' in front of the create-bosh-director.sh command, this is intentional and will cause the script to be run in the current shell.</p>
<p>The creation of the BOSH Director will take a few minutes, and you will be prompted to enter the passphrase for the generation of a public/private rsa key pair.</p>
<p>In case of any issues see <a href="https://github.com/finkit/bosh-on-gcp/terraform/README.md">bosh-on-gcp create BOSH Director</a> docs.</p>
<p>Learn more:
<ul>
<li><a href="https://bosh.io/docs/cli-v2.html#create-env">bosh.io: bosh create-env</a></li>
</ul>
</p>
</div></div>
<div class="section prepare"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>3</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Log in</h2>
</div>
</div>
<p>The final steps of the create-bosh-director.sh script are to create an alias for the BOSH Environment and set the BOSH_ENVIRONMENT, BOSH_CLIENT and BOSH_CLIENT_SECRET environment variables, which make it easier to interact with:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ # No need to execute the following commands</h4>
<h4 class="terminal-code-text">$ # bosh alias-env bosh-director -e 10.0.0.6 --ca-cert <(bosh int ./creds.yml --path /director_ssl/ca)</h4>
<h4 class="terminal-code-text">$ # export BOSH_ENVIRONMENT=bosh-director</h4>
<h4 class="terminal-code-text">$ # export BOSH_CLIENT=admin</h4>
<h4 class="terminal-code-text">$ # export BOSH_CLIENT_SECRET=`bosh int ./creds.yml --path /admin_password`</h4>
</div>
<p>Running bosh env should return details of the BOSH Director, similar to:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh env</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
Name gcpbosh
UUID a5c4d96f-6812-4310-a2e6-a890f8d1aaf2
Version 262.3.0 (00000000)
CPI google_cpi
Features compiled_package_cache: disabled
config_server: disabled
dns: disabled
snapshots: disabled
User admin
Succeeded</h4>
</div>
<p>We are now ready to deploy!</p>
</div></div>
<div class="section deploy"><div class="center-column"><h1>Deploy</h1>
<p>Before we proceed we need to understand what BOSH needs to deploy software.</p>
<h3>What to deploy</h3>
<p>Software that is deployed with BOSH needs to be packaged in a special format called a <b>release</b>. For each service that will be deployed, a release needs to contain source files, configuration files, installation scripts, etc. For example, a redis release would contain the source code for redis, redis configuration defaults and redis init scripts.</p>
<h3>How to deploy</h3>
<p>Each BOSH deployment needs to provide a specially structured configuration file - <b>deployment manifest</b>. This file defines what resources are going to be deployed, what services are going to be running on each of resources and properties that will be passed to services configuration files. For example, for a redis deployment manifest, there are entries for how many and what size redis VMs there should be and how redis should be configured.</p>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/release.html">bosh.io: Release</a></li>
<li><a href="http://bosh.io/docs/deployment.html">bosh.io: Deployment</a></li>
</ul>
</p>
<button onClick="$.fn.fullpage.moveSlideRight();" id="next-button">Next</button>
</div></div>
<div class="section deploy"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>1</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Create BOSH release</h2>
</div>
</div>
<p>We are going to use a simple BOSH release that deploys an http server.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ git clone https://github.com/finkit/learn-bosh-on-gcp-release</h4>
<h4 class="terminal-code-text">$ cd learn-bosh-on-gcp-release</h4>
<h4 class="terminal-code-text">$ bosh create-release</h4>
</div>
<p>Upload generated release to BOSH Director:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh upload-release</h4>
</div>
<p>Check uploaded releases:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh releases</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
Name Version Commit Hash
learn-bosh-on-gcp 0+dev.1 [your commit hash]
1 releases</h4>
</div>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/create-release.html">bosh.io: Creating a Release</a></li>
</ul>
</p>
</div></div>
<div class="section deploy"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>2</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Upload stemcell</h2>
</div>
</div>
<p>A Stemcell is an operating system image that BOSH uses to create VMs. Official BOSH stemcells are maintained with security updates at <a href="https://bosh.io">bosh.io</a>.</p>
<p>Upload stemcell to BOSH Director:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh upload-stemcell https://s3.amazonaws.com/bosh-core-stemcells/google/bosh-stemcell-3431.10-google-kvm-ubuntu-trusty-go_agent.tgz</h4>
</div>
<p>Check uploaded stemcells:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh stemcells</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
Name Version
bosh-warden-google-kvm-ubuntu-trusty-go_agent [your stemcell version]
1 stemcells</h4>
</div>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/stemcell.html">bosh.io: What is a Stemcell?</a></li>
<li><a href="http://bosh.io/stemcells">bosh.io: Full list of available stemcells</a></li>
</ul>
</p>
</div></div>
<div class="section deploy"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>3</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Update cloud config</h2>
</div>
</div>
<p>The newly created BOSH Director will not have any cloud config defined:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh cloud-config</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
No cloud config
Exit code 1</h4>
</div>
<p>Update cloud config on BOSH Director:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh update-cloud-config cloud-config.yml</h4>
</div>
<p>Check cloud config:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh cloud-config</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
azs:
...
Succeeded</h4>
</div>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/cloud-config.html">bosh.io: Cloud Config</a></li>
</ul>
</p>
</div></div>
<div class="section deploy"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>4</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>And deploy</h2>
</div>
</div>
<p>Run deploy by providing path to deployment manifest. Deployment manifest specifies what services to deploy, their properties and resources configuration.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp deploy manifest.yml</h4>
</div>
<p>See the list of deployed instances as it was specified in manifest:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh instances</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
...
Deployment 'learn-bosh-on-gcp'
Instance Process State AZ IPs
learn-bosh-on-gcp/guid running z1 10.0.0.10
1 instances
Succeeded</h4>
</div>
<p>See that our service is up and running.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ curl 10.0.0.10:8080</h4>
<h4 class="terminal-printout-text">Hello, Anonymous from <uuid></h4>
</div>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/deployment-manifest.html">bosh.io: Deployment Manifest Schema</a></li>
<li><a href="http://bosh.io/docs/deploying-step-by-step.html">bosh.io: Deploying Step-by-step</a></li>
</ul>
</p>
</div></div>
<div class="section modify"><div class="center-column"><h1>Modify Deployment</h1>
<p>Now we will update our deployment with new version of software. We will modify some properties. And we are going to scale our deployment.</p>
<button onClick="$.fn.fullpage.moveSectionDown();" id="next-button">Next</button>
</div></div>
<div class="section modify"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>1</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Modify release</h2>
</div>
</div>
<p>BOSH makes it easy to modify and deploy new versions of software. Let's modify our release source files.</p>
<p>In release folder open src/simple_server/app.rb and change the name to yours.</p>
<p>Create new version of release (force option is used to ignore warning about local changes), upload new version of release to the BOSH Director and deploy:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh create-release --force</h4>
<h4 class="terminal-code-text">$ bosh upload-release</h4>
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp deploy manifest.yml</h4>
</div>
<p>See that the updated version was deployed:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ curl 10.0.0.10:8080</h4>
<h4 class="terminal-printout-text">Hello, [your name] from <uuid></h4>
</div>
</div></div>
<div class="section modify"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>2</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Scale deployment</h2>
</div>
</div>
<p>With BOSH it is easy to scale deployments. All you need to do is modify number of instances in manifest file.</p>
<p>Open manifest.yml and change number of instances under instance_groups from 1 to 2. Add another IP to list of static_ips: 10.0.0.11</ma>.</p>
<p>Run deploy:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp deploy manifest.yml</h4>
</div>
<p>Check that 2 instances were deployed:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh instances</h4>
<h4 class="terminal-printout-text">Using environment '10.0.0.6' as client 'admin'
Deployment 'learn-bosh-on-gcp'
Instance Process State AZ IPs
learn-bosh-on-gcp/guid-1 running z1 10.0.0.10
learn-bosh-on-gcp/guid-2 running z1 10.0.0.11
2 instances
Succeeded</h4>
</div>
<p>See that we have 2 instances of our service running:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ curl 10.0.0.10:8080</h4>
<h4 class="terminal-printout-text">Hello, [your name] from <uuid-1></h4>
<h4 class="terminal-code-text">$ curl 10.0.0.11:8080</h4>
<h4 class="terminal-printout-text">Hello, [your name] from <uuid-2></h4>
</div>
</div></div>
<div class="section modify"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>3</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Change properties</h2>
</div>
</div>
<p>Every release can specify a set of properties that need to be set in deployment manifest and provided to service. For example, that can be database credentials, address of another service, etc.</p>
<p>Our release allows to change property port on which server is listening. You can see the list of properties that can be modified in learn-bosh-release/jobs/app/spec. Let's open manifest.yml and under the section properties set the value of port to 8888:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">...
jobs:
- name: app
release: learn-bosh-on-gcp
properties:
port: 8888
...</h4>
</div>
<p>Now we can just re-deploy our manifest changes. Note, we don't need to build new release version, configuration files will be regenerated with new properties:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp deploy manifest.yml</h4>
</div>
<p>Let's see that our property was changed:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ curl 10.0.0.10:8888</h4>
<h4 class="terminal-printout-text">Hello, [your name] from <uuid-1></h4>
<h4 class="terminal-code-text">$ curl 10.0.0.11:8888</h4>
<h4 class="terminal-printout-text">Hello, [your name] from <uuid-1></h4>
</div>
</div></div>
<div class="section recover"><div class="center-column"><h1>When something goes wrong</h1>
<p>BOSH provides a set of recovery mechanisms. Let's break our deployment and find ways to fix it.</p>
<button onClick="$.fn.fullpage.moveSectionDown();" id="next-button">Next</button>
</div></div>
<div class="section recover"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>1</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Failing service</h2>
</div>
</div>
<p>BOSH is using <a href="https://mmonit.com/monit/">monit</a> to monitor running services. If the service goes down it will bring it up. Let's watch how this works. SSH to one of instances:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp ssh app/0</h4>
<h4 class="terminal-code-text">$ sudo -i</h4>
<h4 class="terminal-code-text"># watch monit summary</h4>
<h4 class="terminal-printout-text">The Monit daemon 5.2.5 uptime: 2m
Process 'app' running
System 'system_localhost' running
</h4>
</div>
<p>In a separate window (on host) let's kill our runnning server:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ curl 10.0.0.11:8888/kill</h4>
</div>
<p>Back in the instance window notice that monit will report process as 'Does not exist' and after some period service will be brought back up by monit again.</p>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/vm-monit.html">bosh.io: Process monitoring with Monit</a></li>
</p>
</div></div>
<div class="section recover"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>2</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Failing VM</h2>
</div>
</div>
<p>What if there is a problem with instance that is running our service? BOSH offers manual and automatic recovery when there are problems with infrastructure resources like VMs or disks. In this exercise we are going to kill one of our instances and use manual recovery option.</p>
<p>Lets destroy one of our instances. SSH to BOSH Bastion VM.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ gcloud compute ssh bosh-bastion</h4>
</div>
<p>Delete one of the instances.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ gcloud compute instances delete id-1</h4>
</div>
<p>Let's see that one of the instances is in a bad state:</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh instances</h4>
<h4 class="terminal-printout-text">...
Instance Process State IPs
app/guid-1 running 10.0.0.10
app/guid-2 unresponsive agent 10.0.0.11
...</h4>
</div>
<p>One of the components in BOSH is the Health Monitor. It independently watches system health and will bring missing instances back up by instructing infrastructure to recreate missing resources like VMs with the required persistent disk. Keep running <b>bosh instances</b> and see that instance is brought up and service is running eventually.</p>
<p>Now let's turn off automatic repair and manually resolve the issue.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh update-resurrection off</h4>
</div>
<p>Kill one of the containers again as described above. Run cloud check and select option "Recreate VM and wait for processes to start".</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp cloud-check</h4>
</div>
<p>Cloud check command allows to manually resolve issues when resources (VMs and persistent disks) are in a bad state. Run <b>bosh instances</b> to see all instances running again.</p>
<p>Learn more:
<ul>
<li><a href="http://bosh.io/docs/resurrector.html">bosh.io: Automatic repair with Resurrector</a></li>
<li><a href="http://bosh.io/docs/cck.html">bosh.io: Manual repair with Cloud Check</a></li>
<li><a href="http://bosh.io/docs/vm-config.html">bosh.io: Configuration and file locations of BOSH VMs</a></li>
</p>
</div></div>
<div class="section recover"><div class="center-column"><div class="view-heading">
<div class="circle-container">
<div class="circle-number">
<h2>3</h2>
</div>
</div>
<div class="view-heading-text-container">
<h2>Debugging failing deploy</h2>
</div>
</div>
<p>When deploy command fails there are could be a number of reasons:</p>
<ul>
<li>Invalid network configuration in deployment manifest (e.g. IP address is in use or out of subnet range)</li>
<li>Infrastructure provider failed to create VM or disk (e.g. quota exceeded, instance type is not available)</li>
<li>Properties required by release were not provided in manifest</li>
<p>Let's add another job to our manifest <b>router</b> (it will redirect all requests between servers). Note, that since uploaded release already contains this job, we don't need to update release.</p>
<div class="terminal-block">
<h4 class="terminal-printout-text">- name: router
templates:
- name: router
instances: 1
resource_pool: default
networks:
- name: default
static_ips: [10.0.0.12]</h4>
</div>
<p>Re-deploy with new job.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp deploy manifest.yml</h4>
<h4 class="terminal-printout-warning">...Failed: `router/0 (...)' is not running after update.
</h4>
</div>
<p>Oh-oh, looks like deploy failed. Let's get our service logs, untar them and check stderr log.</p>
<div class="terminal-block">
<h4 class="terminal-code-text">$ bosh -d learn-bosh-on-gcp logs router/0</h4>
</div>
<p>We should find this error: "At least one server must be provided". Router fails to route because there is no servers specified.</p>
<p>Let's add a property to router job to specify our servers pointing to their static IPs and ports.</p>
<div class="terminal-block">
<h4 class="terminal-printout-text">- name: router
templates:
- name: router
instances: 1
resource_pool: default
networks:
- name: default
static_ips: [10.0.0.12]
properties:
servers: "http://10.0.0.10:8888, http://10.0.0.11:8888"</h4>
</div>
<p>Re-deploy and see it succeeds.</p>
<p>Now running <b>curl -L http://10.0.0.10:8888</b> should give us responses from different servers.</p>
</div></div>
<div class="section last"><div class="center-column"><h1>Done!</h1>
<p>In this tutorial we used BOSH to deploy services, updated our deployment with source changes, scaled the number of services and changed their properties. We recovered from failing service, failing VM and failing deploy.</p>
<p>We were using VirtualBox environment with Warden CPI (Cloud Provider Interface). Warden is a Linux container management tool and Warden CPI abstracts VMs as Linux containers. The Director can work with any CPI that implements a certain API to manage IaaS resources. There are several supported CPIs for different IaaS providers: AWS, GCP, Openstack, vSphere and vCloud. Read more about CPIs here: <a href="http://bosh.io/docs/cpi-api-v1.html">http://bosh.io/docs/cpi-api-v1.html</a>.</p>
<p>Learn more:
<ul>
<li><a href="https://bosh.io/docs">bosh.io: Docs</a></li>
</p>
</div></div>
</div>
</body>
</html>