Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Issue #56: Adds vault-ui ingress #65

Merged
Show file tree
Hide file tree
Changes from 10 commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
08e2f9d
Issue #56: Adds vault-ui ingress
SonOfLope Feb 26, 2024
a1dd62e
Issue #56: Adds external-dns annotations
SonOfLope Feb 26, 2024
f2477b6
Issue #56: Fix external-dns annotation
SonOfLope Feb 26, 2024
924538c
Issue #56: Remove https annotation
SonOfLope Feb 26, 2024
658ec4c
Issue #56 : Force https
SonOfLope Feb 26, 2024
063460f
Issue #56: cluster-issuer annotation fix
SonOfLope Feb 26, 2024
af7fa64
Issue #56: change port binding and annotations
SonOfLope Feb 26, 2024
d94ca05
Issue #56: revert changes to working fix
SonOfLope Feb 26, 2024
c558fac
Issue #56: enable tls communication
SonOfLope Feb 26, 2024
aa7ff60
Issue #56: Add vault documentation
SonOfLope Mar 5, 2024
fb601e1
Issue #56: Rename Vault-argoCD-workflow.svg to vault-argocd-workflow.svg
SonOfLope Mar 5, 2024
8a06594
Issue #56: Adds crds for vault config operator. Still needs a fix bec…
SonOfLope Mar 5, 2024
d70afd0
Merge branch 'main' into 56-as-a-developer-i-would-like-an-ingress-to…
SonOfLope Mar 5, 2024
cb9b90e
Issue #56: Delete .github/workflows/test.yml
SonOfLope Mar 5, 2024
135f811
Issue #56: Fix doc deadlink
SonOfLope Mar 5, 2024
474803d
Issue #56: Fixing yaml lint and file name based on convention
SonOfLope Mar 6, 2024
c225432
Issue #56: Fix markdown png reference
SonOfLope Mar 6, 2024
c6ca88b
Issue #56: Updated in order to merge. config operator will be pushed …
SonOfLope Mar 8, 2024
0d6108e
Issue #56: revert value file changes
SonOfLope Mar 8, 2024
305ab4f
Issue #56: update value file name
SonOfLope Mar 8, 2024
5838a45
Merge branch 'main' into 56-as-a-developer-i-would-like-an-ingress-to…
SonOfLope Mar 8, 2024
eb666e2
Issue #56: update vault documentation with argo plugin
SonOfLope Mar 19, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 23 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
name: Python api-test workflows

on:
pull_request:
types:
- opened
- closed
- synchronize

jobs:
lint-test:
uses: ai-cfia/github-workflows/.github/workflows/workflow-lint-test-python.yml@main
secrets: inherit

markdown-check:
uses: ai-cfia/github-workflows/.github/workflows/workflow-markdown-check.yml@main
with:
config-file-path: '.mlc_config.json'
secrets: inherit

repo-standard:
uses: ai-cfia/github-workflows/.github/workflows/workflow-repo-standards-validation.yml@main
secrets: inherit
2 changes: 1 addition & 1 deletion docs/generic-achitecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -158,7 +158,7 @@ flowchart

3. Representation of the two diagrams above in one

![Diagram](png/ha-redundancy-lb.png)
![Diagram](img/ha-redundancy-lb.png)

## References

Expand Down
21 changes: 21 additions & 0 deletions docs/img/Vault-argoCD-workflow.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/create-new-secret.PNG
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
File renamed without changes
Binary file added docs/img/pat-token-scope.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/img/pv-secret-engine.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
89 changes: 89 additions & 0 deletions docs/secret-management.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# Secret management
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Create follow issue to turn this into an ADR to review by team and security.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.


## Introduction

Secrets are sensitive pieces of information that should be protected from
unauthorized access. In the context of a Kubernetes cluster, secrets are used to
store sensitive data such as passwords, tokens, and keys. To allow for secure
and efficient management of secrets, we are using HashiCorp Vault, a tool that
is designed to manage secrets and protect sensitive data. Vault provides a
centralized way to manage access to secrets and encryption keys, and it also has
the ability to generate dynamic secrets on demand. This document provides an
overview of the secret management process and the role of Vault in securing and
managing secrets in the Kubernetes cluster.

## Vault architecture

Vault is a highly available and distributed system that is designed to provide
secure storage and management of secrets. It is built on a client-server
architecture, with the server being the central component that stores and
manages secrets, and the clients being the applications and services that access
the secrets. The server is responsible for authenticating clients, authorizing
access to secrets, and providing encryption and decryption services. The server
is also responsible for generating dynamic secrets on demand, which are
short-lived and are automatically revoked after a certain period of time.

Current configuration allows vault to inject secrets into pods using a sidecar
container that runs the Vault Agent Injector. The Vault Agent Injector is a
mutating webhook that intercepts requests to create or update pods and injects
secrets into the pod's file system. This allows clients (hosted applications) to
access secrets as files, which is a secure and efficient way to manage secrets
in a Kubernetes environment.

The following diagram illustrates the workflow of the Vault Agent Injector and
how developers can manage secrets of hosted applications:
![Developer workflow diagram](img/Vault-argoCD-workflow.svg)

## Secret management process

The secret management process involves the following steps:

1. **Secret creation**: Secrets are created and stored in Vault using the Vault
CLI or API. When a secret is created, it is encrypted and stored in the
central Vault server.

2. **Secret retrieval**: Applications and services can retrieve secrets from
Vault using the Vault CLI or API. When a secret is retrieved, it is
decrypted and returned to the client in a secure manner.

3. **Dynamic secret generation**: Vault has the ability to generate dynamic
secrets on demand. This means that instead of storing static secrets in
Vault, Vault can generate short-lived secrets that are automatically revoked
after a certain period of time. This provides an additional layer of
security and reduces the risk of unauthorized access to secrets.

4. **Access control**: Vault provides fine-grained access control to secrets,
allowing administrators to define policies that specify which clients can
access which secrets. This ensures that only authorized clients can access
sensitive data. Currently, we are using the Kubernetes authentication method
to authenticate hosted applications and authorize access to secrets. As for
the human users, we are using the Github authentication method to
authenticate and authorize access to secrets.

## Create, read, update, and delete secrets

Vault provides a UI service to manage secrets. The UI service is a web-based
user interface that allows administrators to create, read, update, and delete
secrets. The service also provides a way to manage access control policies
and audit logs. The service is accessible through a web browser and is
protected by the same security mechanisms as the Vault server.

### Steps

1. In order to gain access to the Vault UI service, you need to have the
appropriate permissions and access to the Vault URL. It is currently
configured to give access to any member of the `ai-cfia` organization on
Github.
2. Generate a personal access token on Github and use it to authenticate to the
Vault UI service. The scope of the token should be : ![PAT token
scope](img/pat-token-scope.png)
3. Gain access to the Vault UI service by navigating to the Vault URL in a web
browser. You will be prompted to authenticate using your Github PAT token.
4. Once authenticated, you will be able to create, read, update, and delete
secrets using the UI service. Simply navigate to the PV secret engine and
follow the path to your applications secrets. The PV secret engine is a
key-value store that allows you to store and manage secrets for your
applications. ![PV secret engine](img/pv-secret-engine.png)
5. Once in the directory of your application secrets, simply click on 'create
new version' and you will be able to add, update, or delete secrets as
needed. ![Create mew secret](img/create-new-secret.PNG)
2 changes: 1 addition & 1 deletion kubernetes/aks/system/vault/helm/values.yml
Original file line number Diff line number Diff line change
Expand Up @@ -788,7 +788,7 @@ server:
plugin_directory = "/vault/plugins"

listener "tcp" {
tls_disable = true
tls_disable = false
address = "[::]:8200"
cluster_address = "[::]:8201"
tls_cert_file = "/vault/userconfig/vault-ha-tls/vault.crt"
Expand Down
32 changes: 32 additions & 0 deletions kubernetes/aks/system/vault/ingress.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: vault-ui
namespace: vault
labels:
app.kubernetes.io/name: vault-ui
app.kubernetes.io/instance: vault
annotations:
external-dns.alpha.kubernetes.io/target: inspection.alpha.canada.ca
cert-manager.io/cluster-issuer: letsencrypt-prod
nginx.ingress.kubernetes.io/rewrite-target: /
nginx.ingress.kubernetes.io/backend-protocol: "HTTPS"
kubernetes.io/tls-acme: "true"
spec:
ingressClassName: nginx
tls:
- hosts:
- "vault.inspection.alpha.canada.ca"
secretName: vault-ui
rules:
- host: "vault.inspection.alpha.canada.ca"
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: vault-ui
port:
name: https