Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Use Rails credentials instead of deprecated and removed secrets #553

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion developer_setup/seeding_test_inventory.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ ems = ManageIQ::Providers::Amazon::CloudManager.create!(
We can find these values by looking at the amazon provider's vcr cassette file,
`src/manageiq/manageiq-providers-amazon/spec/vcr_cassettes/manageiq/providers/amazon/cloud_manager/refresher_inventory_object.yml`

We can see that `'us-east-1'` is used as the region in the URIs, and in the `config/secrets.defaults.yml`
We can see that `'us-east-1'` is used as the region in the URIs, and in the Rails credentials
and `spec/factories/ext_management_system.rb` files we can see that `AMAZON_CLIENT_ID` and
`AMAZON_CLIENT_SECRET` are used for the userid and password values for the authentication.

Expand Down
1 change: 0 additions & 1 deletion providers/subclassing_an_existing_provider.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ Initialized empty Git repository in /home/grare/adam/src/manageiq/manageiq/plugi
create bin/update
create bundler.d
create bundler.d/.keep
create config/secrets.defaults.yml
create config/settings.yml
create lib/manageiq-providers-awesome_private_cloud.rb
create lib/manageiq/providers/awesome_private_cloud/engine.rb
Expand Down
71 changes: 54 additions & 17 deletions providers/writing_vcr_specs.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,34 +22,71 @@ The next thing we have to take care of is hiding "secrets". Since the VCR YAML

VCR handles this with the `config.define_cassette_placeholder` option. You provide VCR with a string that you want to be replaced, and then what you want it to be replaced with. This allows for hostnames / passwords / etc... to be used when recording the cassette but the values will not be written to the resulting YAML files.

ManageIQ has a pattern to help you with this, simply create a `config/secrets.defaults.yml` file:
ManageIQ has a pattern to help you with this. We use rails credentials. Run the following command in the main application directory:

```
EDITOR=vi be rails credentials:edit --help
```

This will provide more information about rails credentials.

A sample workflow for VCR cassettes would be to edit the test environment's credentials and use them in test.

Run the prior command without the help option and with the 'test' environment:

```
EDITOR=vi be rails credentials:edit --environment test
```

You can use your preferred editor by specifying it on the command line.

In the editor, set your credentials, for example:

```yaml
---
test:
awesome_cloud_defaults: &awesome_cloud_defaults
awesome_cloud:
access_key: AWESOME_CLOUD_ACCESS_KEY
secret_key: AWESOME_CLOUD_SECRET_KEY
awesome_cloud:
<<: *awesome_cloud_defaults
```

Then create a `config/secrets.yml` file (this file will not be committed and should be in your .gitignore):
```yaml
---
test:
awesome_cloud:
access_key: "YOUR_REAL_ACCESS_KEY"
secret_key: "YOUR_REAL_SECRET_KEY"
After saving, this will create or update:
* a plain text key file if you haven't already created one: config/credentials/test.key
* an encrypted credentials file: config/credentials/test.yml.enc

Add memoized methods in spec/spec_helper.rb with the defaults you want to assumed when running from cassettes to avoid leaking actual credentials:

```ruby
def credentials_awesome_cloud_host
@credentials_awesome_cloud_host ||= Rails.application.credentials.awesome_cloud_host || "awesome-cloud-host"
end
Comment on lines +59 to +61
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We didn't have a "host" in the old examples (this was modeled after a public-cloud so no hostname needed)


def credentials_awesome_cloud_user
@credentials_awesome_cloud_user ||= Rails.application.credentials.awesome_cloud_user || "awesome-cloud-user"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be Rails.application.credentials.awesome_cloud&.access_key

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I'm trying to generalize them. Does that make sense for a provider that uses user/password authentication? If so, I can do that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Well two things, one I think even if there was a user: "..." it would be Rails.application.credentials.awesome_cloud.user not Rails.application.credentials.awesome_cloud_user and two the example here has access_key not user

I don't think I'd want to change the "writing a provider" example to have host/user/password because that doesn't match with the rest of the example code (there's an example connect method and an example api_client, etc...)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe in addition to editing this specific example to work with the new credentials interface we need a "recording_vcrs.md" guide that covers how to do this more generally?

end

def credentials_awesome_cloud_password
@credentials_autosde_password ||= Rails.application.credentials.awesome_cloud_password || "change_me"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rails.application.credentials.awesome_cloud&.secret_key

end
```

Then add the following to your `VCR.configure` block in `spec/spec_helper.rb` after setting the `config.cassette_library_dir`:


```ruby
secrets = Rails.application.secrets
secrets.awesome_cloud.each do |key, val|
config.define_cassette_placeholder(secrets.awesome_cloud_defaults[key]) { val }
defaults = {
"host_key" => credentials_awesome_cloud_host,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"host_key" => credentials_awesome_cloud_host,

"access_key" => credentials_awesome_cloud_user,
"secret_key" => credentials_awesome_cloud_password
}

defaults.each do |key, value|
config.define_cassette_placeholder(value) do
Rails.application.credentials.dig(:awesome_cloud, key)
end
end
```


### Writing the tests

Now that we have VCR configured it is time to start writing your spec tests. First we will start with the Refresher to test the refresh process of your new provider.
Expand All @@ -62,7 +99,7 @@ describe ManageIQ::Providers::AwesomeCloud::CloudManager::Refresher do
let(:zone) { EvmSpecHelper.create_guid_miq_server_zone.last }
let!(:ems) do
FactoryBot.create(:ems_awesome_cloud, :zone => zone).tap do |ems|
access_key, secret_key = Rails.application.secrets.awesome_cloud.values_at(:access_key, :secret_key)
access_key, secret_key = credentials_awesome_cloud_user, credentials_awesome_cloud_password

ems.update_authentication(:default => {:userid => access_key, :password => secret_key})
end
Expand Down Expand Up @@ -125,4 +162,4 @@ rm spec/vcr_cassettes/manageiq/providers/awesome_cloud/cloud_manager/refresher.y
bundle exec rspec spec/models/manageiq/awesome_cloud/cloud_manager/refresher_spec.rb
```

Make sure that you have your `config/secrets.yml` file still present, you might have to update the expected counts as things in your environment have likely changed but you now should have an updated VCR cassette.
Make sure that you have your rails credentials still present. You might have to update the expected counts as things in your environment have likely changed but you now should have an updated VCR cassette.