Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Doc from openstack admin guide #1083

Merged
merged 42 commits into from
Dec 5, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
5ca6b83
Add openstack projects & users management doc
seunghun1ee May 22, 2024
6b83576
Add horizon customisation doc
seunghun1ee May 23, 2024
ac8bcba
Add ceph management doc
seunghun1ee May 23, 2024
e5b7a77
Add ceph operation doc
seunghun1ee May 28, 2024
f4b2630
Add openstack operation docs
seunghun1ee May 28, 2024
8f30483
Add wazuh operation docs
seunghun1ee May 28, 2024
7b29ccd
Add hardware inventory management doc
seunghun1ee May 28, 2024
3dc7a2c
Move advise tool intro
seunghun1ee May 29, 2024
21aaa5a
Add baremetal node management doc
seunghun1ee May 29, 2024
5482670
Add gpu doc
seunghun1ee May 29, 2024
05cff81
Fix errors and add to index
seunghun1ee May 29, 2024
4fced04
Remove repeating section
seunghun1ee May 30, 2024
879f8dc
Add more instruction for ADVise tool
seunghun1ee May 30, 2024
2ddc2a2
Fix formatting
seunghun1ee May 30, 2024
828f42c
Merge drive replacement related sections into one
seunghun1ee Jun 14, 2024
a092cba
Reference Cephadm & Kayobe doc as deployment guide
seunghun1ee Jun 14, 2024
5cf8890
Merge Wazuh documents
seunghun1ee Jun 17, 2024
342a4a2
Update old contents
seunghun1ee Jun 17, 2024
e5a0f50
Attach Release Train document for more info
seunghun1ee Jun 17, 2024
d170a9e
Remove baremetal management doc
seunghun1ee Jun 17, 2024
a2833f5
Fix formatting
seunghun1ee Jun 17, 2024
124a2a3
Adding missing /
seunghun1ee Jul 9, 2024
e12e8fa
Update content to Antelope and misc changes
seunghun1ee Sep 12, 2024
7d41f5b
Update Cephadm playbook info
seunghun1ee Sep 16, 2024
109ca13
Replace etc/kayobe to $KAYOBE_CONFIG_PATH
seunghun1ee Sep 16, 2024
8edf08f
specify keyring is populated
seunghun1ee Oct 9, 2024
20e46a3
Add rebooting case
seunghun1ee Oct 9, 2024
275ce2c
Remove missing document
seunghun1ee Nov 4, 2024
c24dc1d
Make hardware inventory doc bifrost specific
seunghun1ee Nov 4, 2024
5b12bd0
Add reference to monitoring doc
seunghun1ee Nov 4, 2024
b7b776f
Use reboot playbook rather than shutdown command
seunghun1ee Nov 4, 2024
f7018d5
Use env variable
seunghun1ee Nov 4, 2024
9e324ba
Make Vault and Openstack reconfig doc refer each other
seunghun1ee Nov 4, 2024
b9cefdf
Fix: Use RST syntax of Note
seunghun1ee Nov 4, 2024
106b14f
Update to use some of upstream doc
seunghun1ee Nov 4, 2024
15b575f
Better wordings on section intro
seunghun1ee Nov 7, 2024
7703f93
Remove unnecessary curly brackets
seunghun1ee Nov 7, 2024
3eb6537
Add note of reconfiguring monitoring service
seunghun1ee Nov 7, 2024
8f001c4
Fix spacing
seunghun1ee Nov 7, 2024
869fb57
Remove command prefixes
seunghun1ee Nov 7, 2024
1788f7b
Add warning of brief downtime
seunghun1ee Nov 7, 2024
a6872b5
Remove outdated information
seunghun1ee Dec 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 5 additions & 3 deletions doc/source/configuration/cephadm.rst
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
====
Ceph
====
.. _cephadm-kayobe:

================
Cephadm & Kayobe
================

This section describes how to use the Cephadm integration included in StackHPC
Kayobe configuration to deploy Ceph.
Expand Down
14 changes: 7 additions & 7 deletions doc/source/configuration/ci-cd.rst
Original file line number Diff line number Diff line change
Expand Up @@ -57,26 +57,26 @@ Runner Deployment
Ideally an Infra VM could be used here or failing that the control host.
Wherever it is deployed the host will need access to the :code:`admin_network`, :code:`public_network` and the :code:`pulp registry` on the seed.

2. Edit the environment's :code:`${KAYOBE_CONFIG_PATH}/environments/${KAYOBE_ENVIRONMENT}/inventory/groups` to add the predefined :code:`github-runners` group to :code:`infra-vms`
2. Edit the environment's :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/inventory/groups` to add the predefined :code:`github-runners` group to :code:`infra-vms`

.. code-block:: ini

[infra-vms:children]
github-runners

3. Edit the environment's :code:`${KAYOBE_CONFIG_PATH}/environments/${KAYOBE_ENVIRONMENT}/inventory/hosts` to define the host(s) that will host the runners.
3. Edit the environment's :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/inventory/hosts` to define the host(s) that will host the runners.

.. code-block:: ini

[github-runners]
prod-runner-01

4. Provide all the relevant Kayobe :code:`group_vars` for :code:`github-runners` under :code:`${KAYOBE_CONFIG_PATH}/environments/${KAYOBE_ENVIRONMENT}/inventory/group_vars/github-runners`
4. Provide all the relevant Kayobe :code:`group_vars` for :code:`github-runners` under :code:`$KAYOBE_CONFIG_PATH/environments/$KAYOBE_ENVIRONMENT/inventory/group_vars/github-runners`
* `infra-vms` ensuring all required `infra_vm_extra_network_interfaces` are defined
* `network-interfaces`
* `python-interpreter.yml` ensuring that `ansible_python_interpreter: /usr/bin/python3` has been set

5. Edit the ``${KAYOBE_CONFIG_PATH}/inventory/group_vars/github-runners/runners.yml`` file which will contain the variables required to deploy a series of runners.
5. Edit the ``$KAYOBE_CONFIG_PATH/inventory/group_vars/github-runners/runners.yml`` file which will contain the variables required to deploy a series of runners.
Below is a core set of variables that will require consideration and modification for successful deployment of the runners.
The number of runners deployed can be configured by removing and extending the dict :code:`github-runners`.
As for how many runners present three is suitable number as this would prevent situations where long running jobs could halt progress other tasks whilst waiting for a free runner.
Expand Down Expand Up @@ -120,7 +120,7 @@ Runner Deployment

7. If the host is an actual Infra VM then please refer to upstream `Infrastructure VMs <https://docs.openstack.org/kayobe/latest/configuration/reference/infra-vms.html>`__ documentation for additional configuration and steps.

8. Run :code:`kayobe playbook run ${KAYOBE_CONFIG_PATH}/ansible/deploy-github-runner.yml`
8. Run :code:`kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/deploy-github-runner.yml`

9. Check runners have registered properly by visiting the repository's :code:`Action` tab -> :code:`Runners` -> :code:`Self-hosted runners`

Expand All @@ -130,9 +130,9 @@ Runner Deployment
Workflow Deployment
-------------------

1. Edit :code:`${KAYOBE_CONFIG_PATH}/inventory/group_vars/github-writer/writer.yml` in the base configuration making the appropriate changes to your deployments specific needs. See documentation for `stackhpc.kayobe_workflows.github <https://github.com/stackhpc/ansible-collection-kayobe-workflows/tree/main/roles/github>`__.
1. Edit :code:`$KAYOBE_CONFIG_PATH/inventory/group_vars/github-writer/writer.yml` in the base configuration making the appropriate changes to your deployments specific needs. See documentation for `stackhpc.kayobe_workflows.github <https://github.com/stackhpc/ansible-collection-kayobe-workflows/tree/main/roles/github>`__.

2. Run :code:`kayobe playbook run ${KAYOBE_CONFIG_PATH}/ansible/write-github-workflows.yml`
2. Run :code:`kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/write-github-workflows.yml`

3. Add all required secrets and variables to repository either via the GitHub UI or GitHub CLI (may require repository owner)

Expand Down
4 changes: 2 additions & 2 deletions doc/source/configuration/lvm.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,6 @@ hosts:

.. code-block:: console

mkdir -p ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/pre.d
cd ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/pre.d
mkdir -p $KAYOBE_CONFIG_PATH/hooks/overcloud-host-configure/pre.d
cd $KAYOBE_CONFIG_PATH/hooks/overcloud-host-configure/pre.d
ln -s ../../../ansible/growroot.yml 30-growroot.yml
2 changes: 2 additions & 0 deletions doc/source/configuration/monitoring.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
Monitoring
==========

.. _monitoring-service-configuration:

Monitoring Configuration
========================

Expand Down
2 changes: 1 addition & 1 deletion doc/source/configuration/release-train.rst
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
.. _stackhpc_release_train:
.. _stackhpc-release-train:

======================
StackHPC Release Train
Expand Down
4 changes: 2 additions & 2 deletions doc/source/configuration/swap.rst
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,6 @@ hosts:

.. code-block:: console

mkdir -p ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/post.d
cd ${KAYOBE_CONFIG_PATH}/hooks/overcloud-host-configure/post.d
mkdir -p $KAYOBE_CONFIG_PATH/hooks/overcloud-host-configure/post.d
cd $KAYOBE_CONFIG_PATH/hooks/overcloud-host-configure/post.d
ln -s ../../../ansible/swap.yml 10-swap.yml
5 changes: 5 additions & 0 deletions doc/source/configuration/vault.rst
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
.. _hashicorp-vault:

================================
Hashicorp Vault for internal PKI
================================
Expand Down Expand Up @@ -111,6 +113,9 @@ Certificates generation
Create the external TLS certificates (testing only)
---------------------------------------------------

This method should only be used for testing. For external TLS on production systems,
See `Installing External TLS Certificates <installing-external-tls-certificates>`__.

Typically external API TLS certificates should be generated by a organisation's trusted internal or third-party CA.
For test and development purposes it is possible to use Vault as a CA for the external API.

Expand Down
71 changes: 52 additions & 19 deletions doc/source/configuration/wazuh.rst
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,20 @@
Wazuh
=====

`Wazuh <https://wazuh.com>`_ is a security monitoring platform.
It monitors for:

* Security-related system events.
* Known vulnerabilities (CVEs) in versions of installed software.
* Misconfigurations in system security.

The short version
=================

#. Create an infrastructure VM for the Wazuh manager, and add it to the wazuh-manager group
#. Configure the infrastructure VM with kayobe: ``kayobe infra vm host configure``
#. Edit your config under
``etc/kayobe/inventory/group_vars/wazuh-manager/wazuh-manager``, in
``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/wazuh-manager``, in
particular the defaults assume that the ``provision_oc_net`` network will be
used.
#. Generate secrets: ``kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/wazuh-secrets.yml``
Expand All @@ -27,14 +34,14 @@ Provisioning an infra VM for Wazuh Manager.
Kayobe supports :kayobe-doc:`provisioning infra VMs <deployment.html#infrastructure-vms>`.
The following configuration may be used as a guide. Config for infra VMs is documented :kayobe-doc:`here <configuration/reference/infra-vms>`.

Add a Wazuh Manager host to the ``wazuh-manager`` group in ``etc/kayobe/inventory/hosts``.
Add a Wazuh Manager host to the ``wazuh-manager`` group in ``$KAYOBE_CONFIG_PATH/inventory/hosts``.

.. code-block:: ini

[wazuh-manager]
os-wazuh

Add the ``wazuh-manager`` group to the ``infra-vms`` group in ``etc/kayobe/inventory/groups``.
Add the ``wazuh-manager`` group to the ``infra-vms`` group in ``$KAYOBE_CONFIG_PATH/inventory/groups``.

.. code-block:: ini

Expand All @@ -43,7 +50,7 @@ Add the ``wazuh-manager`` group to the ``infra-vms`` group in ``etc/kayobe/inven
[infra-vms:children]
wazuh-manager

Define VM sizing in ``etc/kayobe/inventory/group_vars/wazuh-manager/infra-vms``:
Define VM sizing in ``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/infra-vms``:

.. code-block:: yaml

Expand All @@ -57,7 +64,7 @@ Define VM sizing in ``etc/kayobe/inventory/group_vars/wazuh-manager/infra-vms``:
# Capacity of the infra VM data volume.
infra_vm_data_capacity: "200G"

Optional: define LVM volumes in ``etc/kayobe/inventory/group_vars/wazuh-manager/lvm``.
Optional: define LVM volumes in ``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/lvm``.
``/var/ossec`` often requires greater storage space, and ``/var/lib/wazuh-indexer``
may be beneficial too.

Expand All @@ -79,7 +86,7 @@ may be beneficial too.
create: true


Define network interfaces ``etc/kayobe/inventory/group_vars/wazuh-manager/network-interfaces``:
Define network interfaces ``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/network-interfaces``:

(The following is an example - the names will depend on your particular network configuration.)

Expand All @@ -91,7 +98,7 @@ Define network interfaces ``etc/kayobe/inventory/group_vars/wazuh-manager/networ


The Wazuh manager may need to be exposed externally, in which case it may require another interface.
This can be done as follows in ``etc/kayobe/inventory/group_vars/wazuh-manager/network-interfaces``,
This can be done as follows in ``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/network-interfaces``,
with the network defined in ``networks.yml`` as usual.

.. code-block:: yaml
Expand Down Expand Up @@ -183,7 +190,7 @@ Deploying Wazuh Manager services
Setup
-----

To install a specific version modify the wazuh-ansible entry in ``etc/kayobe/ansible/requirements.yml``:
To install a specific version modify the wazuh-ansible entry in ``$KAYOBE_CONFIG_PATH/ansible/requirements.yml``:

.. code-block:: yaml

Expand All @@ -204,7 +211,7 @@ Edit the playbook and variables to your needs:
Wazuh manager configuration
---------------------------

Wazuh manager playbook is located in ``etc/kayobe/ansible/wazuh-manager.yml``.
Wazuh manager playbook is located in ``$KAYOBE_CONFIG_PATH/ansible/wazuh-manager.yml``.
Running this playbook will:

* generate certificates for wazuh-manager
Expand All @@ -214,7 +221,7 @@ Running this playbook will:
* setup and deploy wazuh-dashboard on wazuh-manager vm
* copy certificates over to wazuh-manager vm

Wazuh manager variables file is located in ``etc/kayobe/inventory/group_vars/wazuh-manager/wazuh-manager``.
Wazuh manager variables file is located in ``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/wazuh-manager``.

You may need to modify some of the variables, including:

Expand All @@ -225,31 +232,38 @@ You may need to modify some of the variables, including:

If you are using multiple environments, and you need to customise Wazuh in
each environment, create override files in an appropriate directory,
for example ``etc/kayobe/environments/production/inventory/group_vars/``.
for example ``$KAYOBE_CONFIG_PATH/environments/production/inventory/group_vars/``.

Files which values can be overridden (in the context of Wazuh):

- etc/kayobe/inventory/group_vars/wazuh/wazuh-manager/wazuh-manager
- etc/kayobe/wazuh-manager.yml
- etc/kayobe/inventory/group_vars/wazuh/wazuh-agent/wazuh-agent
- $KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh/wazuh-manager/wazuh-manager
- $KAYOBE_CONFIG_PATH/wazuh-manager.yml
- $KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh/wazuh-agent/wazuh-agent

You'll need to run ``wazuh-manager.yml`` playbook again to apply customisation.

Secrets
-------

Wazuh secrets playbook is located in ``etc/kayobe/ansible/wazuh-secrets.yml``.
Wazuh requires that secrets or passwords are set for itself and the services with which it communiticates.
Wazuh secrets playbook is located in ``$KAYOBE_CONFIG_PATH/ansible/wazuh-secrets.yml``.
Running this playbook will generate and put pertinent security items into secrets
vault file which will be placed in ``$KAYOBE_CONFIG_PATH/wazuh-secrets.yml``.
If using environments it ends up in ``$KAYOBE_CONFIG_PATH/environments/<env_name>/wazuh-secrets.yml``
Remember to encrypt!

Wazuh secrets template is located in ``etc/kayobe/ansible/templates/wazuh-secrets.yml.j2``.
Wazuh secrets template is located in ``$KAYOBE_CONFIG_PATH/ansible/templates/wazuh-secrets.yml.j2``.
It will be used by wazuh secrets playbook to generate wazuh secrets vault file.


.. code-block:: console

kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/wazuh-secrets.yml

.. note:: Use ``ansible-vault`` to view the secrets:

``ansible-vault view --vault-password-file ~/vault.password $KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-manager/wazuh-secrets.yml``

Configure Wazuh Dashboard's Server Host
---------------------------------------

Expand Down Expand Up @@ -366,7 +380,7 @@ Verification
------------

The Wazuh portal should be accessible on port 443 of the Wazuh
manager’s IPs (using HTTPS, with the root CA cert in ``etc/kayobe/ansible/wazuh/certificates/wazuh-certificates/root-ca.pem``).
manager’s IPs (using HTTPS, with the root CA cert in ``$KAYOBE_CONFIG_PATH/ansible/wazuh/certificates/wazuh-certificates/root-ca.pem``).
The first login should be as the admin user,
with the opendistro_admin_password password in ``$KAYOBE_CONFIG_PATH/wazuh-secrets.yml``.
This will create the necessary indices.
Expand All @@ -378,9 +392,9 @@ Logs are in ``/var/log/wazuh-indexer/wazuh.log``. There are also logs in the jou
Wazuh agents
============

Wazuh agent playbook is located in ``etc/kayobe/ansible/wazuh-agent.yml``.
Wazuh agent playbook is located in ``$KAYOBE_CONFIG_PATH/ansible/wazuh-agent.yml``.

Wazuh agent variables file is located in ``etc/kayobe/inventory/group_vars/wazuh-agent/wazuh-agent``.
Wazuh agent variables file is located in ``$KAYOBE_CONFIG_PATH/inventory/group_vars/wazuh-agent/wazuh-agent``.

You may need to modify some variables, including:

Expand All @@ -390,6 +404,25 @@ Deploy the Wazuh agents:

``kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/wazuh-agent.yml``

The Wazuh Agent is deployed to all hosts in the ``wazuh-agent``
inventory group, comprising the ``seed`` group
plus the ``overcloud`` group (containing all hosts in the
OpenStack control plane).

.. code-block:: ini

[wazuh-agent:children]
seed
overcloud

The hosts running Wazuh Agent should automatically be registered
and visible within the Wazuh Manager dashboard.

.. note:: It is good practice to use a `Kayobe deploy hook
<https://docs.openstack.org/kayobe/latest/custom-ansible-playbooks.html#hooks>`_
to automate deployment and configuration of the Wazuh Agent
following a run of ``kayobe overcloud host configure``.

Verification
------------

Expand Down
6 changes: 3 additions & 3 deletions doc/source/contributor/pre-commit.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,12 +29,12 @@ Once done you should find `pre-commit` is available within the `kayobe` virtuale

To run the playbook using the following command

- ``kayobe playbook run ${KAYOBE_CONFIG_PATH}/ansible/install-pre-commit-hooks.yml``
- ``kayobe playbook run $KAYOBE_CONFIG_PATH/ansible/install-pre-commit-hooks.yml``

Whereas to run the playbook when control host bootstrap runs ensure it registered as symlink using the following command

- ``mkdir -p ${KAYOBE_CONFIG_PATH}/hooks/control-host-bootstrap/post.d``
- ``ln -s ${KAYOBE_CONFIG_PATH}/ansible/install-pre-commit-hooks.yml ${KAYOBE_CONFIG_PATH}/hooks/control-host-bootstrap/post.d/install-pre-commit-hooks.yml``
- ``mkdir -p $KAYOBE_CONFIG_PATH/hooks/control-host-bootstrap/post.d``
- ``ln -s $KAYOBE_CONFIG_PATH/ansible/install-pre-commit-hooks.yml $KAYOBE_CONFIG_PATH/hooks/control-host-bootstrap/post.d/install-pre-commit-hooks.yml``

All that remains is the installation of the hooks themselves which can be accomplished either by
running `pre-commit run` or using `git commit` when you have changes that need to be committed.
Expand Down
Loading