Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Vault preferres HCL #18

Open
linuxmail opened this issue Nov 4, 2022 · 5 comments · May be fixed by #26
Open

Vault preferres HCL #18

linuxmail opened this issue Nov 4, 2022 · 5 comments · May be fixed by #26

Comments

@linuxmail
Copy link

Hello,

Hashicorp wants to use HCL instead of JSON for configuration. There is also a comment about this problem also here and I've added to a comment here too.

I had to switch from the Debian package to the Archive method, which is not a problem but it took a bit time, to understand, why it wasn't working. After the switch and getting most parts up and running, I tried to add service registration to Consul, but the JSON wasn't parsed correctly.
It seems, it wasn't fixed correctly, as I still get:

2022-11-04T11:35:23.879+0100 [WARN]  unknown or unsupported field consul found in configuration at /etc/vault/config.json
2022-11-04T11:35:23.879+0100 [INFO]  proxy environment: http_proxy="" https_proxy="" no_proxy=""
2022-11-04T11:35:23.887+0100 [INFO]  core: Initializing version history cache for core
2022-11-04T11:35:23.889+0100 [INFO]  core: raft retry join initiated

with:

    extra_config      => {
      cluster_addr => "https://${::fqdn}:8201",
      api_addr     => "https://${::fqdn}:8200",
      service_registration =>  {
         consul            => { 
          address           => "${::fqdn}:8501",
          token             => $consul_agent_token,
          scheme            => https,
          tls_cert_file     => $tls_cert_file,
          tls_min_version   => 'tls12',
          tls_cipher_suites => 'TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_RSA_WITH_AES_256_GCM_SHA384,TLS_RSA_WITH_AES_128_GCM_SHA256',
          service_tags      => "vault",
        }
      }
    },

It works, but the message "unknown or unsupported field consul found" is still there. And also the question is .. how to handle the systemd file, when install_method package / repository is used.

cu denny
@avitacco
Copy link

avitacco commented Mar 27, 2023 •
edited
Loading

I created my own vault module to manage our vault instances at my job, which outputs HCL. I will get a PR started to add HCL output to this module. Some of my code is pretty opinionated, so hopefully, @bastelfreak doesn't mind too much and we can merge the two modules into one.

Edit: Did some end-of-the-day work and this is my very rough non-working first pass at merging my work into this module. https://github.com/avitacco/puppet-vault

@avitacco avitacco linked a pull request Mar 28, 2023 that will close this issue
Open
@alaunay
Copy link

alaunay commented Jan 8, 2024

It's even worse, I just tried to install, spent 2 hours not understanding why my parameters from the module weren't used -- there is a /etc/vault.d/vault.hcl which is used and the config.json is completely ignored.

If you try to delete the vault.hcl file, vault refuses to run:
Jan 8 14:54:58 vault systemd[1]: vault.service - "HashiCorp Vault - A tool for managing secrets" was skipped because of an unmet condition check (ConditionFileNotEmpty=/etc/vault.d/vault.hcl).

It's /lib/systemd/system/vault.service has a condition to forbid it to run if the hcl file is not present...
ConditionFileNotEmpty=/etc/vault.d/vault.hcl

As is, the puppet-vault module is totally unusable.

@bastelfreak
Copy link
Member

The problem with HCL is that, as far as I know, there's no native way in Ruby nor Puppet to generate it. For JSON, we can just use a Puppet Hash and throw it into to_json_pretty(). That makes it quite simple to manage it.

As is, the puppet-vault module is totally unusable.

To be fair that's not true. it depends which package you're using. If your source add the condition I suggest to use a systemd::dropin_file that clears ConditionFileNotEmpty or adjusts the file ending.

If someone has an idea for easy HCL generation I'm happy to talk about it.

@alaunay
Copy link

alaunay commented Jan 8, 2024

Agreed, my comment was a bit harsh, sorry. Probably a result of the lost time.

The source is the debian repo for hashicorp, configured by this module via the hashi_stack module, so nothing fancy here, just the last official source :/

root@vault:~# apt-cache policy vault
vault:
Installed: 1.15.4-1
Candidate: 1.15.4-1
Version table:
*** 1.15.4-1 500
500 https://apt.releases.hashicorp.com bookworm/main amd64 Packages
100 /var/lib/dpkg/status

In order to use the config.json...

root@vault:/etc/vault.d# ls -l
total 8
-rwxr-x--- 1 vault vault 276 Jan 8 13:20 config.json
-rw-r--r-- 1 vault vault 0 Dec 5 02:49 vault.env
-rw-r--r-- 1 vault vault 1053 Jan 8 14:54 vault.hcl.orig

Commenting out the condition gives:

Jan 8 15:39:40 vault vault[12899]: error loading configuration from /etc/vault.d/vault.hcl: stat /etc/vault.d/vault.hcl: no such file or directory

... aaaand I found out why.

I use:
vault::install_method: 'repo'

Which gives in manifests/config.pp:
'repo': { $real_manage_service_file = false }

So...
vault::manage_service_file: true

Jan 8 15:59:58 vault (vault)[13955]: vault.service: Failed to locate executable /usr/local/bin/vault: No such file or directory

So:
vault::bin_dir: '/usr/bin'

Did the final trick.

For the HCL, besides a ERB template (like the ntp module), I have no idea.

@edrude
Copy link

edrude commented Aug 8, 2024

Agreed, my comment was a bit harsh, sorry. Probably a result of the lost time.

The source is the debian repo for hashicorp, configured by this module via the hashi_stack module, so nothing fancy here, just the last official source :/

root@vault:~# apt-cache policy vault vault: Installed: 1.15.4-1 Candidate: 1.15.4-1 Version table: *** 1.15.4-1 500 500 https://apt.releases.hashicorp.com bookworm/main amd64 Packages 100 /var/lib/dpkg/status

In order to use the config.json...

root@vault:/etc/vault.d# ls -l total 8 -rwxr-x--- 1 vault vault 276 Jan 8 13:20 config.json -rw-r--r-- 1 vault vault 0 Dec 5 02:49 vault.env -rw-r--r-- 1 vault vault 1053 Jan 8 14:54 vault.hcl.orig

Commenting out the condition gives:

Jan 8 15:39:40 vault vault[12899]: error loading configuration from /etc/vault.d/vault.hcl: stat /etc/vault.d/vault.hcl: no such file or directory

... aaaand I found out why.

I use: vault::install_method: 'repo'

Which gives in manifests/config.pp: 'repo': { $real_manage_service_file = false }

So... vault::manage_service_file: true

Jan 8 15:59:58 vault (vault)[13955]: vault.service: Failed to locate executable /usr/local/bin/vault: No such file or directory

So: vault::bin_dir: '/usr/bin'

Did the final trick.

For the HCL, besides a ERB template (like the ntp module), I have no idea.

This worked for me on RHEL 9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Merging in code from my private module avitacco/puppet-vault
5 participants
@bastelfreak @linuxmail @alaunay @edrude @avitacco