Disclaimer
This is NOT an official Cisco application and comes with absolute NO WARRANTY!
Please check the LICENSE for further information.
Credits
This demo has been largely inspired by the great work done by Ned Bellavance for his 'Getting Started' and 'Deep Dive' Terraform courses on Pluralsight. Although it has been mostly reworked and extended to work with Intersight and Vault, some constructs and strategies have been reused in this repository.
The scope of this project is to demonstrate how Cisco Intersight can fit an existing CI/CD environment with centralized secret management and multiple teams sharing resources.
This repository contains the code and the instructions to setup the whole demo on your laptop.
At the end of the setup you will have:
- Two GitHub private repositories, one for the Compute team, one for the Security Team
- A Jenkins instance, preconfigured with two pipelines and the necessary credentials and plugins
- A Vault instance configured to authenticate on behalf of Consul. This Vault instance will also store your Intersight credentials
- A Consul instance that stores both compute and security configuration parameters and remote state
Notes
- Once the setup is complete, you can "pause" it to free resources with
minikube stop - You can resume the demo setup with
minikube start. However, you are required to recreate the port-forwarding rules as well as Vault configuration. Since we are running in dev mode, once the pod restarts, it losts all data. (see below to recreate the port-forwarding rules and Vault configuration) - The first time you run the pipelines, it may take up to 20 minutes as Jenkins will need to download the container agent images for the first time. Subsequent runs should happen much faster. The same is true with the installation of Vault, Consul and Jenkins as minikube will need to download the images first
Once the demo environment is up and running, connect to Jenkins and execute the pipelines to deploy the demo code to Intersight.
The security-team pipeline can only be executed after the compute-team pipeline, as it will leverage its remote Terraform state file.
The demo environment supports whetever Terraform code you have in your SCM repository, you are definitely encouraged to fork this repo and create additional use cases!
- Docker Desktop - Ensure you reserve at least 4GB Memory for Docker Desktop
- minikube
- helm
- awk
- git
- terraform
- consul client
- vault client
- An Intersight secret file (get it from intersight.com, settings -> API keys )
- A GitHub personal access token
Warning: This demo will create policies and a server profile. This profile will be associated to an existing server.
In Intersight, identify a server you can use and take note of its management IP.
Warning: READ before you copy/paste! Some steps require manual population of environment variables based on your setup
From a terminal, clone this repository
git clone https://github.com/rtortori/iatc-intersight-iac.git
Move to the repo directory and delete the existing repository
cd iatc-intersight-iac && rm -rf .git
Copy your Intersight Secret in the demo-setup/intersight directory (replace "/path/to" with the actual directory)
cp /path/to/SecretKey.txt demo-setup/intersight/SecretKey.txt
In the 'team-compute/2_main/3_datasources.tf' file, replace "192.168.30.22" the the management IP of the server you want to associate the server profile to
export SERVER_MGT_IP="server.mgt.ip.address"
sed -i '' -e "s/192.168.30.22/$SERVER_MGT_IP/g" team-compute/2_main/3_datasources.tf
Set the Intersight API key as an environment variable
export INTERSIGHT_API_TOKEN="your_token_here"
Set your GitHub token and username as an environment variable
(E.g. GITHUB_USERNAME="rtortori")
export GITHUB_TOKEN="your_token_here"
export GITHUB_USERNAME="myuser"
Start minikube
minikube start
Deploy Consul
helm repo add hashicorp https://helm.releases.hashicorp.com
helm repo update
helm install -f ./demo-setup/consul/consul_values.yaml consul hashicorp/consul --version 0.29.0
Deploy Jenkins
helm repo add jenkins https://charts.jenkins.io
helm repo update
helm install -f ./demo-setup/jenkins/jenkins_values.yaml jenkins jenkins/jenkins --version 3.1.2
Deploy Vault
helm repo add hashicorp https://helm.releases.hashicorp.com
helm install vault hashicorp/vault --version 0.8.0 \
--set "server.dev.enabled=true" \
--set "injector.enabled=false" \
--set "ui.enabled=true"
Ensure all pods are in the 'Running' state. They also need to show '1/1'
If some pods are not ready yet, just wait a few more minutes and retry the command. It may take some time depending on your internet connection
kubectl get pods
Create the port-forwarding rules to access Jenkins, Vault and Consul
kubectl port-forward service/consul-consul-ui 8500:80 --address 0.0.0.0 >/dev/null&
kubectl port-forward service/jenkins 8080:8080 >/dev/null&
kubectl port-forward service/vault-ui 8200:8200 >/dev/null&
Set Environment variables to access deployed resources (Vault token will be 'root' as we run it in Dev mode)
export CONSUL_HTTP_TOKEN=$(kubectl get secret consul-consul-bootstrap-acl-token -o=jsonpath={.data.token} | base64 -d)
export JENKINS_URL="http://127.0.0.1:8080"
export JENKINS_USERNAME="admin"
export JENKINS_PASSWORD="$(kubectl exec --namespace default -it svc/jenkins -c jenkins -- /bin/cat /run/secrets/chart-admin-password && echo)"
export COOKIE_JAR=/tmp/cookies
export VAULT_TOKEN=root
export VAULT_ADDR='http://0.0.0.0:8200'
Consul Setup
cd demo-setup/consul
terraform init
terraform plan -out consul.tfplan
terraform apply --auto-approve consul.tfplan
Create Intersight Compute and Security Configuration in Consul
consul kv put intersight_compute/configuration/compute_config @compute_config.json
consul kv put intersight_security/configuration/security_config @security_config.json
Create an ACL token for Vault to manage Consul Authentication
export GLOB_MGT_TOKEN=$(consul acl token create -policy-name=global-management | grep SecretID | awk '{print $2}')
Enable the consul secret engine in Vault
vault secrets enable consul
Configure Vault to authenticate to Consul
vault write consul/config/access \
address=consul-consul-server:8500 \
token=$GLOB_MGT_TOKEN
Configure compute and security roles in Vault and map to Consul Policies
vault write consul/roles/intersight_compute policies=intersight_compute
vault write consul/roles/intersight_security policies=intersight_security
Change Directory to the intersight directory
cd ../intersight
Write Intersight secrets to Vault
vault secrets enable -path=intersight kv
vault kv put intersight/credentials/cert [email protected]
vault kv put intersight/credentials/api_key api_key=$INTERSIGHT_API_TOKEN
Configure credentials in Jenkins. Set Jenkins crumbs
JENKINS_CRUMB=$(curl --silent --cookie-jar $COOKIE_JAR $JENKINS_URL'/crumbIssuer/api/xml?xpath=concat(//crumbRequestField,":",//crumb)' -u $JENKINS_USERNAME:$JENKINS_PASSWORD)
Set GitHub and Vault Credentials in Jenkins.
For this demo we will use the same token, but in production you will have policies and different tokens
# GH credentials
curl -X POST "http://$JENKINS_USERNAME:[email protected]:8080/credentials/store/system/domain/_/createCredentials" --cookie $COOKIE_JAR -H $JENKINS_CRUMB \
--data-urlencode 'json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "GitHub_Credentials",
"username": "dummy",
"password": "'"$GITHUB_TOKEN"'",
"description": "GH Credentials",
"$class": "com.cloudbees.plugins.credentials.impl.UsernamePasswordCredentialsImpl"
}
}'
# Vault Compute Credentials
curl -X POST "http://$JENKINS_USERNAME:[email protected]:8080/credentials/store/system/domain/_/createCredentials" --cookie $COOKIE_JAR -H $JENKINS_CRUMB \
--data-urlencode 'json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "Intersight_Compute_Vault",
"secret": "root",
"description": "Vault token for Compute team",
"$class": "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl"
}
}'
# Vault Security Credentials
curl -X POST "http://$JENKINS_USERNAME:[email protected]:8080/credentials/store/system/domain/_/createCredentials" --cookie $COOKIE_JAR -H $JENKINS_CRUMB \
--data-urlencode 'json={
"": "0",
"credentials": {
"scope": "GLOBAL",
"id": "Intersight_Security_Vault",
"secret": "root",
"description": "Vault token for Security team",
"$class": "org.jenkinsci.plugins.plaincredentials.impl.StringCredentialsImpl"
}
}'
You will now need to login to Jenkins. Paste the following command and connect with your browser
echo "Login to Jenkins at $JENKINS_URL\nUsername: $JENKINS_USERNAME \nPassword: $JENKINS_PASSWORD"
Once connected, go to:
- "Manage Jenkins" -> "Global Tool Configuration" -> "Terraform" -> "Add Terraform" -> Set "Name" = terraform
- "Install from bintray.com" -> Select "Terraform 0.13.5 linux (amd64)" -> Hit "Save"
Switch back to your terminal and move to the github directory
cd ../github
Configure your GitHub token in the terraform configuration file
sed -i '' -e "s/my_gh_token/$GITHUB_TOKEN/g" 2_variables.tf
Create the GitHub repositories and the Jenkins pipeline
terraform init
terraform plan
terraform apply -auto-approve
Move to the team-compute directory
cd ../../team-compute
Init the Compute repository and push to GitHub
git init
echo ".terraform/" >> .git/info/exclude
echo "*.tfstate" >> .git/info/exclude
echo "*.tfstate.backup" >> .git/info/exclude
git add .
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/$GITHUB_USERNAME/intersight_iac_demo_compute.git
git push -u origin main
Move to the team-security directory
cd ../team-security
Init the Security repository and push to GitHub
git init
echo ".terraform/" >> .git/info/exclude
echo "*.tfstate" >> .git/info/exclude
echo "*.tfstate.backup" >> .git/info/exclude
git add .
git commit -m "first commit"
git branch -M main
git remote add origin https://github.com/$GITHUB_USERNAME/intersight_iac_demo_security.git
git push -u origin main
The Jenkinsfile(s) in both pipelines include most of the variables used by Terraform.
You may want to customize the following - or not :)
- TF_VAR_resource_prefix: Each resource created in Intersight will have this prefix
- TV_VAR_organization: The name of the Intersight organization you want your resources in. Most likely is going to be 'default' but in some cases you may want to use an alternative organization
To clean-up objects created in Intersight, just run the security and the compute pipelines (in this order) and when it asks for confirmation, click on 'Abort'. It will execute a terraform destroy -auto-approve and everything will disappear.
To clean-up the GitHub repositories and delete all traces of this demo
cd demo-setup/github && terraform destroy -auto-approve
minikube stop
minikube delete
unset INTERSIGHT_API_TOKEN
unset GITHUB_TOKEN
unset GITHUB_USERNAME
unset CONSUL_HTTP_TOKEN
unset JENKINS_USERNAME
unset JENKINS_PASSWORD
unset JENKINS_CRUMB
unset COOKIE_JAR
unset VAULT_TOKEN
unset VAULT_ADDR
unset GLOB_MGT_TOKEN