Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add diagrams of provisioning methods #3069

Merged
merged 10 commits into from
Sep 2, 2024
Merged

Add diagrams of provisioning methods #3069

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
2a8b220
Add image sources for provisioning methods
Lennonka Jul 20, 2024
ec2d08f
Remove outdated image source
Lennonka Jul 20, 2024
944295a
Diagram previews
Lennonka Jul 20, 2024
8845e35
Address feedback
Lennonka Aug 8, 2024
5e9a138
Update previews
Lennonka Aug 8, 2024
d430e40
Neutralize Pulp
Lennonka Aug 8, 2024
95f9632
Update previews
Lennonka Aug 8, 2024
c057553
fixup
Lennonka Aug 8, 2024
0bbc745
Drop DHCP from fullhost bootdisk
Lennonka Aug 12, 2024
3a9be6f
Incorporate suggestions from Ewoud
Lennonka Aug 13, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added guides/common/images/provisioning-image-cloudinit.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added guides/common/images/provisioning-image-finish.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added guides/common/images/provisioning-installer-bootdisk-fullhost.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added guides/common/images/provisioning-installer-bootdisk-subnet.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added guides/common/images/provisioning-installer-httpboot.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added guides/common/images/provisioning-installer-pxeboot.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added guides/common/images/provisioning-template-vmware-userdata.png
View file
Open in desktop
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
16 changes: 0 additions & 16 deletions guides/common/images/user-data-sequence-satellite.txt
View file
Open in desktop

This file was deleted.

48 changes: 48 additions & 0 deletions guides/image-sources/foreman.pstyle
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
' Foreman style for PlantUML diagrams
' This style was created for sequence diagrams and may be incomplete.

skinparam RoundCorner 8
skinparam Shadowing false

' #ffcc32 Foreman yellow
' #025d8c Foreman blue

skinparam sequence {
ArrowColor #025d8c
LifeLineBorderColor #ecf4f8
LifeLineBorderThickness 32
DividerBorderColor #025d8c
DividerFontColor #025d8c
group {
TitleFontColor #025d8c
BorderColor #d2e4ed
FontColor #025d8c
}
groupHeader {
BackgroundColor #d2e4ed
FontColor #025d8c
}
}

skinparam Actor {
BackgroundColor #025d8c
BorderColor #025d8c
FontColor #025d8c
}

skinparam Participant {
BackgroundColor #025d8c
BorderColor #025d8c
FontColor White
}

skinparam Note {
BackgroundColor #d2e4ed
BorderColor #025d8c
FontColor #025d8c
}

' Puppet enabled=1, disabled=0
!$puppet = 0

autonumber
16 changes: 16 additions & 0 deletions guides/image-sources/prov-create-host-pxe.iuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
User -> Foreman : clicks **Create Host**
Foreman -> Proxy : requests free IP address
Proxy -> DHCP : reads DHCP cofiguration
note over Foreman : pings IP address
User -> Foreman : submits the **Create host** form
group Templates [*PXE* installer boot]
Foreman -> Proxy : renders bootloader configuration files
end
Proxy -> TFTP : deploys bootloader configuration files
Foreman -> Proxy : commands to download installation media
note over Proxy : downloads installation media
Foreman -> Proxy : creates DHCP reservation
Proxy -> DHCP : forwards DHCP reservation
Foreman -> Proxy : creates DNS records
Proxy -> DNS : forwards DNS records
note over Foreman : host is created\n(build mode enabled)
4 changes: 4 additions & 0 deletions guides/image-sources/prov-first-local-boot-hdd.iuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
!if ($networkboot)
Host -> DHCP : requests the reserved IP
!endif
Host -> Host : boots from HDD
5 changes: 5 additions & 0 deletions guides/image-sources/prov-first-local-boot-pxe.iuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
Host -> DHCP : requests the reserved IP and DHCP options
Host -> TFTP : downloads bootloader
Host -> TFTP : gets MAC-based bootloader config
Host -> Host : config tells bootloader to boot from HDD
Host -> Host : boots from HDD
8 changes: 8 additions & 0 deletions guides/image-sources/prov-initial-configuration.iuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
!if ($puppet)
opt when Puppet enabled
Host -> Puppet : sends CSR
Puppet -> Host : sends certificate (if allowed to)
end
!else
note over Host : initial host configuration\n- remote execution,\n- signs Puppet client cert,\n- runs Ansible roles,\n- etc.
!endif
6 changes: 6 additions & 0 deletions guides/image-sources/prov-installation-media.iuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
loop
Host -> Foreman : requests installation media
Foreman -> Proxy : requests installation media
Proxy -> Foreman : provides installation media
Foreman -> Host : provides installation media
end
50 changes: 50 additions & 0 deletions guides/image-sources/provisioning-image-cloudinit.plantuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,50 @@
@startuml

!include foreman.pstyle

title Image-based provisioning with cloud-init configuration

actor User
participant "Provisioned\nInstance" as Host
participant "Foreman" as Foreman
participant "Foreman\nProxy" as Proxy
participant "Infrastructure\nCloud" as Cloud
participant DNS
!if ($puppet)
participant "Puppet\nserver" as Puppet
!endif

note over Foreman : has an image\nwith the *User Data* flag

== Create host in Foreman ==

User -> Foreman : clicks **Create Host**
User -> Foreman : selects cloud resource,\nthe image and submits
group Template [cloud-init]
Foreman -> Proxy : renders the cloud-init script
end
Comment on lines +23 to +25
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Internally Foreman will render the user data template as the first step:
https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/compute.rb#L63-L66

This rendered template is then sent to the compute resource directly and the whole Proxy is not involved at all.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this is correct then.

Foreman -> Cloud : creates new instance\nwith cloud-init info
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is correct: https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/compute.rb#L67-L68

Copy link
Contributor Author

@Lennonka Lennonka Aug 13, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So this?

Suggested change
Foreman -> Cloud : creates new instance\nwith cloud-init info
Foreman -> Cloud : sets up the compute instance\nwith cloud-init info

Cloud -> Foreman : reports IP address
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically it can report IPv4 and IPv6. If both are reported, that also means the DNS record will get both A and AAAA records:
https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/compute.rb#L69-L72

After this will store the instance details that the compute resource provided:
https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/compute.rb#L73-L74

Not all compute resources provide IPs and some provide MAC addresses. In that case it will also try to reserve an IP:
https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/compute.rb#L75-L78

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically it can report IPv4 and IPv6.

That is not the level of detail we want here.

After this will store the instance details that the compute resource provided
Not all compute resources provide IPs and some provide MAC addresses. In that case it will also try to reserve an IP

I don't understand what it means for the diagram.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That is not the level of detail we want here.

It's about singular vs plural.

Suggested change
Cloud -> Foreman : reports IP address
Cloud -> Foreman : reports IP addresses

I don't understand what it means for the diagram.

Probably not for this specific diagram.

Foreman -> Proxy : creates DNS records
Proxy -> DNS : forwards DNS records
Comment on lines +28 to +29
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not entirely correct. First the code reference: https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/dns.rb#L63-L71

It will create all feasible DNS record types. That can mean A, AAAA and PTR (for both v4 and v6), depending on the known IP addresses. So the first part is simplified, but correct.

The second part is IMHO confusing. The Proxy tells the DNS server to create the DNS records. A Foreman Proxy, especially the DNS module, implements the facade pattern. It abstracts away the actual implementation away from Foreman. IMHO it is forwarding the DNS record creation, not the records themselves.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So what wording do you suggest precisely?

!if ($puppet)
Foreman -> Proxy : creates Puppet sign request (autosign enabled)
Proxy -> Puppet : forwards Puppet sign request
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No proxying is happening. The Smart Proxy (depending on implementation) sets up the Puppetserver so it will accept the Certificate Signing Request (CSR) that the client will later present.

In the default Smart Proxy implementation that means Foreman Proxy rewrites autosign.conf but there's also a different one based on providing a JWT in the CSR.

https://theforeman.org/manuals/3.11/index.html#4.3.7PuppetCA documents both puppetca_hostname_whitelisting (default) and puppetca_token_whitelisting on a fairly low level. There's no documentation for this in the new docs nor any documentation on how to set up the more secure token-based implementation. (And on a related note, I noticed I forgot to drop puppetca_puppet_cert documentation so I opened theforeman/theforeman.org#2168.)

!endif
Foreman -> Cloud : starts the instance
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Referencing the code: https://github.com/theforeman/foreman/blob/e80d5558767bb713c13264e821122baea577af76/app/models/concerns/orchestration/compute.rb#L79-L82

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm assuming that means that it's correct.

note over Host : boots up

== Cloud-init script ==

!include prov-initial-configuration.iuml
Host -> Foreman : calls home\n(disables build mode)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the part that can go through the Proxy's template feature. The use of the Foreman Proxy templates module is optional. If that isn't used then cloud-init requests it directly from Foreman.


!if ($puppet)
== First Puppet run ==

!include puppet-run.iuml
!endif

note over Host : in operation

@enduml
47 changes: 47 additions & 0 deletions guides/image-sources/provisioning-image-finish.plantuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
@startuml

!include foreman.pstyle

title Image-based provisioning with Finish script configuration

actor User
participant "Provisioned\nInstance" as Host
participant "Foreman" as Foreman
participant "Foreman\nProxy" as Proxy
participant "Infrastructure\nCloud" as Cloud
participant DNS
!if ($puppet)
participant "Puppet\nserver" as Puppet
!endif

note over Foreman : has an image with credentials\ndefined in compute resource

== Create host in Foreman ==

User -> Foreman : clicks on **Create Host**
User -> Foreman : selects cloud resource, the image and submits
group Template [Finish]
Foreman -> Proxy : renders the Finish script
end
Foreman -> Cloud : creates new instance
Cloud -> Foreman : reports IP address
Foreman -> Proxy : creates DNS records
Proxy -> DNS : forwards DNS records
Foreman -> Cloud : starts the instance
note over Host : boots up
Foreman -> Host : executes Finish script via SSH

== Finish script ==

!include prov-initial-configuration.iuml
Host -> Foreman : calls home\n(disables build mode)

!if ($puppet)
== First Puppet run ==

!include puppet-run.iuml
!endif

note over Host : in operation

@enduml
66 changes: 66 additions & 0 deletions guides/image-sources/provisioning-installer-bootdisk-fullhost.plantuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
@startuml

!include foreman.pstyle
!$networkboot = 0

title Installer-based provisioning with full-host bootdisk

actor User
participant "Provisioned\nHost" as Host
participant "Foreman" as Foreman
participant "Foreman\nProxy" as Proxy
'participant "Pulp\n(Katello)" as Pulp
'participant TFTP
'participant DHCP
participant DNS
!if ($puppet)
participant "Puppet\nserver" as Puppet
!endif

note over Host : powered off

== Create host in Foreman ==

User -> Foreman : clicks **Create Host**
User -> Foreman : assigns static IP address
User -> Foreman : submits the **Create host** form
group Templates [*PXE* installer boot]
Foreman -> Proxy : renders bootloader configuration files
end
Proxy -> Proxy : deploys bootloader configuration files
Foreman -> Proxy : commands to download installation media
note over Proxy : downloads installation media
Foreman -> Proxy : creates DNS records
Proxy -> DNS : forwards DNS records
note over Foreman : host is created\n(build mode enabled)

User -> Foreman : downloads the bootdisk ISO of the host
note over User : writes the bootdisk ISO\nto a USB/CD/DVD drive

== Boot into OS installer ==

User -> Host : configures the machine to boot\nfrom the USB/CD/DVD drive
User -> Host : powers on the machine
Host -> Host : uses IP from the bootdisk
note over Host : boots from the USB/CD/DVD drive
note over Host : loads OS installer
User -> Host : eliminates the USB/CD/DVD drive\n(too soon?)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we don't know the exact step where it happens, do we have to mention it?

Copy link
Contributor Author

@Lennonka Lennonka Aug 8, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well, they have to do it at some point, otherwise they would be stuck in a loop :D
So we might as well suggest when the earliest good time is.

As it is a necessary step in the process, I would prefer to keep it.

!include prov-installation-media.iuml
note over Host : OS is installed
!include prov-initial-configuration.iuml
Host -> Foreman : calls home\n(disables build mode)
note over Host : reboots

== First local boot ==

!include prov-first-local-boot-hdd.iuml

!if ($puppet)
== First Puppet run ==

!include puppet-run.iuml
!endif

note over Host : in operation

@enduml
60 changes: 60 additions & 0 deletions guides/image-sources/provisioning-installer-bootdisk-subnet.plantuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
@startuml

!include foreman.pstyle
!$networkboot = 1

title Installer-based provisioning with subnet bootdisk

actor User
participant "Provisioned\nHost" as Host
participant "Foreman" as Foreman
participant "Foreman\nProxy" as Proxy
'participant "Pulp\n(Katello)" as Pulp
'participant TFTP
participant DHCP
participant DNS
!if ($puppet)
participant "Puppet\nserver" as Puppet
!endif

note over Host : powered off

User -> Foreman : downloads the bootdisk of the subnet
note over User : writes the bootdisk\nto a USB/CD/DVD drive

== Boot into OS installer ==

User -> Host : configures the machine to boot\nfrom the USB/CD/DVD drive
User -> Host : powers on the machine
Host -> DHCP : requests the reserved IP
note over Host : boots from the USB/CD/DVD drive
note over Host : bootloader loads
User -> Host : eliminates the USB/CD/DVD drive\n(too soon?)
Host -> Proxy : gets MAC-based bootloader config
Host -> Proxy : downloads OS installer kernel\nand initial RAM disk
note over Host : OS installer loads
Host -> Proxy : requests installer configuration
group Template [Provision]
Proxy -> Proxy : renders installer configuration
end
Proxy -> Host : gets installer configuration
!include prov-installation-media.iuml
note over Host : OS is installed
!include prov-initial-configuration.iuml
Host -> Foreman : calls home\n(disables build mode)

note over Host : reboots

== First local boot ==

!include prov-first-local-boot-hdd.iuml

!if ($puppet)
== First Puppet run ==

!include puppet-run.iuml
!endif

note over Host : in operation

@enduml
61 changes: 61 additions & 0 deletions guides/image-sources/provisioning-installer-httpboot.plantuml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,61 @@
@startuml

!include foreman.pstyle

title Installer-based provisioning with HTTP boot

actor User
participant "Provisioned\nHost" as Host
participant "Foreman" as Foreman
participant "Foreman\nProxy" as Proxy
'participant "Pulp\n(Katello)" as Pulp
participant TFTP
participant DHCP
participant DNS
!if ($puppet)
participant "Puppet\nserver" as Puppet
!endif

note over Host : powered off

== Create host in Foreman ==

!include prov-create-host-pxe.iuml

== Boot into OS installer ==

User -> Host : powers on the machine
Host -> DHCP : requests the reserved IP and DHCP options
Host -> Proxy : downloads bootloader
note over Host : bootloader loads
Host -> Proxy : gets MAC-based bootloader config
Host -> Proxy : downloads OS installer kernel\nand initial RAM disk
note over Host : OS installer loads
Host -> Proxy : requests installer configuration
group Template [Provision]
Proxy -> Proxy : renders installer configuration
end
Proxy -> Host : receives installer configuration
!include prov-installation-media.iuml
note over Host : OS is installed
!include prov-initial-configuration.iuml
Host -> Foreman : calls home\n(disables build mode)
group Templates [*PXE* local boot]
Foreman -> Proxy : renders bootloader configuration files
end
Proxy -> TFTP : deploys bootloader configuration files
note over Host : reboots

== First local boot ==

!include prov-first-local-boot-pxe.iuml

!if ($puppet)
== First Puppet run ==

!include puppet-run.iuml
!endif

note over Host : in operation

@enduml
Loading