Skip to content

Commit

Permalink
fix: Small fixes for template init
Browse files Browse the repository at this point in the history
  • Loading branch information
keskiju committed Jun 4, 2018
1 parent 2a812b0 commit 3b15daa
Show file tree
Hide file tree
Showing 3 changed files with 92 additions and 103 deletions.
40 changes: 21 additions & 19 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,15 @@ Table of contents:
* [Links](#links)
* [Prerequisites](#prerequisites)
* [Important](#important)
* [Quick start](#quick-start)
* [Upgrading WordPress](#upgrading-wordpress)
* [Local development](#local-development)
* [Deployment](#deployment)
* [Version control](#version-control)
* [Configuration](#configuration)

## Links

> Non-production basic auth credentials: TODO user / painipaini. If the admin account is shared among people, you can find the admin credentials from a shared password manager.
Non-production basic auth credentials: TODO: change `user` / `painipaini`. If the admin account is shared among people, you can find the admin credentials from a shared password manager.

[//]: # (GENERATED LINKS START)

Expand All @@ -36,7 +37,7 @@ Table of contents:
* [Logs (prod)](https://console.cloud.google.com/logs/viewer?project=gcloud-temp1&minLogLevel=0&expandAll=false&resource=container%2Fcluster_name%2Fkube1%2Fnamespace_id%2Fwordpress-template-prod)
* [Logs (stag)](https://console.cloud.google.com/logs/viewer?project=gcloud-temp1&minLogLevel=0&expandAll=false&resource=container%2Fcluster_name%2Fkube1%2Fnamespace_id%2Fwordpress-template-stag)
* [Project documentation](https://github.com/taitounited/wordpress-template/wiki)
* [Uptime monitoring (Stackdriver)](https://app.google.stackdriver.com/uptime?project=gcloud-temp1)
* [Uptime monitoring (Stackdriver)](https://app.google.stackdriver.com/uptime?project=gcloud-temp1)

[//]: # (GENERATED LINKS END)

Expand All @@ -45,34 +46,39 @@ Table of contents:
## Prerequisites

* [docker-compose](https://docs.docker.com/compose/install/)
* [node.js](https://nodejs.org/)
* [taito-cli](https://github.com/TaitoUnited/taito-cli#readme)

## Important

It is recommended to do most modifications in local dev environment first. Use the production environment only for making frequent live modifications like creating new blog posts and managing users.

If the production database contains some confidential data like personally identifiable information of customers, you should never take a full database dump of production data for development purposes. However, if most modifications are made in local development environment and committed to git, there should be no need for production data at all. You can use the staging environment to make sure that the modifications made in local development environment work also with the current production data.
If the production database contains some confidential data like personally identifiable information of customers, you should never take a full database dump of production data for development purposes. Or if you do, data should anonymized carefully. However, if most modifications are made in local development environment and committed to git, there should be no need for production data at all. You can use the staging environment to make sure that the modifications made in local development environment work also with the current production data.

## Upgrading WordPress

Upgrade WordPress version both in `docker-compose.yaml` and in `scripts/heml.yaml`. Push the change to different environment branches. Use the staging environment to check that the upgrade doesn't break anything, before pushing the change to production.

## Local development

> Try to synchronize your work with other developers to avoid conflicts. You can easily overwrite changes of another developer when you save your local data to git.
> Try to synchronize your work with other developers to avoid conflicts. You can easily overwrite changes of another developer when you push your local database changes to git.
> NOTE: Support for remote development environment might be coming later (see README_remote.md)
> Support for remote development environment might be coming later (see README_remote.md)
Install some libraries on host (add `--clean` for clean reinstall):

taito install

# TODO gitignored 'wordpress/data' should also be deleted on --clean
# TODO: gitignored 'wordpress/data' should also be deleted on --clean

Start containers (add `--clean` for clean rebuild and db init using `database/init/*`):

taito start

Show user accounts and other information that you can use to log in:

taito info

Open app in browser:

taito open app
Expand All @@ -81,10 +87,6 @@ Open admin GUI in browser:

taito open admin

Show user accounts and other information that you can use to log in:

taito info

Access database:

taito db connect # access using a command-line tool
Expand Down Expand Up @@ -119,7 +121,7 @@ Cleaning:

taito clean:wordpress # TODO
taito clean:database # TODO
taito clean:data # TODO Clean gitignored wp data
taito clean:data # TODO: Clean gitignored wp data
taito clean:npm # Delete node_modules directories
taito clean # Clean everything

Expand Down Expand Up @@ -153,10 +155,10 @@ Deploying to different environments:
* staging: Merge changes from dev branch to staging branch using fast-forward.
* prod: Merge changes from staging branch to master branch using fast-forward. Version number and release notes are generated automatically by the CI/CD tool.

> NOTE: You can use taito-cli to [manage environment branches](#version-control).
> You can use taito-cli to [manage environment branches](#version-control).
> TODO: Automation for data/db migrations.
> TODO: Command for copying data from production to staging.
TODO: Automation for data/db migrations.
TODO: Command for copying data from production to staging.

## Version control

Expand Down Expand Up @@ -236,7 +238,7 @@ You can use any of the following types in your commit message. Use at least type
Done:
* [ ] GitHub settings
* [ ] Basic project settings
* [ ] Server environments: dev
* [ ] Server environments: stag
* [ ] Server environments: prod

### GitHub settings
Expand All @@ -249,7 +251,7 @@ Options:

Branches:
* Default branch: dev
* Protected branch: master (TODO more protection settings)
* Protected branch: master (TODO: more protection settings)

Collaborators & teams:
* Teams: Select admin permission for the Admins team
Expand All @@ -268,11 +270,11 @@ Collaborators & teams:
Creating a new server environment:

* For a production environment: Configure correct IP on DNS record.
* For a production environment: Configure app url in `taito-config.sh` and hostname in `scripts/wordpress/helm-prod.yaml` file. (TODO taito-config.sh should suffice)
* For a production environment: Configure app url in `taito-config.sh` and hostname in `scripts/wordpress/helm-prod.yaml` file. (TODO: taito-config.sh should suffice)
* Run `taito env apply:ENV` to create an environment. Use the same basic auth credentials for all environments. Basic auth credentials don't have to be strong, but still do not reuse the same password for multiple projects. Update the basic auth username/password to the `package.json` file and to the beginning of this README, if they are not up-to-date.
* Deploy wordpress to the environment either by pushing some changes to the environment branch or by triggering the deployment manually: `taito deployment trigger:ENV`.
* Immediately generate a new password for the admin user by using the WordPress admin GUI (`taito open admin:ENV`). The initial admin password is: `initial-password-change-it-on-wp-admin-immediately`. If the admin account is shared, save the new password to a shared password manager. And never use the same admin password for every environment, as dev database is committed to git.
* TODO Connect persistent volume disk to a separate vm dedicated for file access. In development, rsync files also to a storage bucket for easier access?
* TODO: Connect persistent volume disk to a separate vm dedicated for file access. In development, rsync files also to a storage bucket for easier access?

> Operations on production and staging environments require admin rights, if they contain confidential data. Please contact devops personnel.
Expand Down
152 changes: 70 additions & 82 deletions scripts/taito-template/init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -24,24 +24,16 @@
: "${template_project_path:?}"
: "${mode:?}"

# Determine sed options
if [ "$(uname)" = "Darwin" ]; then
sedi="-i ''"
else
sedi="-i"
fi

# Remove .gitignore to allow committing data to git
rm wordpress/data/.gitignore

# Remove MIT license
# TODO leave a reference to the original?
rm LICENSE
# grep -v '"license":' < package.json > package.json.tmp
# mv package.json.tmp package.json

# Replace repository url in package.json
sed ${sedi} -- "s|TaitoUnited/wordpress-template.git|${taito_organization}/${taito_repo_name}.git|g" package.json
sed -i -- \
"s|TaitoUnited/wordpress-template.git|${taito_organization}/${taito_repo_name}.git|g" package.json

# Add some do not modify notes
echo "Adding do not modify notes..."
Expand All @@ -50,7 +42,7 @@ echo "Adding do not modify notes..."
{
sed '/TEMPLATE NOTE START/q' README.md
echo
echo "> This file has been copied from \
echo "This file has been copied from \
[orig-template](https://github.com/TaitoUnited/orig-template/). Keep \
modifications minimal and improve the original instead. Project \
specific documentation is located in PROJECT.md."
Expand All @@ -61,24 +53,14 @@ truncate --size 0 README.md
cat temp > README.md

# Add 'do not modify' note to readme of helm chart
# echo \
# "> NOTE: This helm chart has been copied from \
# [orig-template](https://github.com/TaitoUnited/orig-template/). It is \
# located here only to avoid accidental build breaks. Do not modify it. \
# Improve the original instead." | \
# cat - scripts/helm/README.md > temp && \
# truncate --size 0 scripts/helm/README.md && \
# cat temp > scripts/helm/README.md

# Add 'do not modify' note to readme of terraform
echo \
"> NOTE: These terraform scripts have been copied from \
[orig-template](https://github.com/TaitoUnited/orig-template/). They are \
located here only to avoid accidental build breaks. Do not modify them. \
Improve the originals instead." | \
cat - scripts/terraform/README.md > temp && \
truncate --size 0 scripts/terraform/README.md && \
cat temp > scripts/terraform/README.md
"> NOTE: This helm chart has been copied from \
[orig-template](https://github.com/TaitoUnited/orig-template/). It is \
located here only to avoid accidental build breaks. Do not modify it. \
Improve the original instead." | \
cat - scripts/helm/README.md > temp && \
truncate --size 0 scripts/helm/README.md && \
cat temp > scripts/helm/README.md

# Add 'do not modify' note to cloudbuild.yaml
printf \
Expand All @@ -95,6 +77,7 @@ echo "Give a short version of the project name '${taito_project}'."
echo "It should be unique but also descriptive as it will be used"
echo "as a database name and as a database user name."
echo
taito_project_short="${taito_project_short}"
while \
[[ -z "${taito_project_short}" ]] || \
[[ "${#taito_project_short}" -gt 10 ]]
Expand All @@ -104,73 +87,78 @@ do
done

echo "Replacing project and company names in files. Please wait..."
if [ "$(uname)" = "Darwin" ]; then
find . -type f -exec sed -i '' \
-e "s/wptemplate/${taito_project_short}/g" 2> /dev/null {} \;
find . -type f -exec sed -i '' \
-e "s/wordpress_template/${taito_repo_name_alt}/g" 2> /dev/null {} \;
find . -type f -exec sed -i '' \
-e "s/wordpress-template/${taito_repo_name}/g" 2> /dev/null {} \;
find . -type f -exec sed -i '' \
-e "s/companyname/${taito_company}/g" 2> /dev/null {} \;
find . -type f -exec sed -i '' \
-e "s/orig-template/wordpress-template/g" 2> /dev/null {} \;
else
find . -type f -exec sed -i \
-e "s/wptemplate/${taito_project_short}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/wordpress_template/${taito_repo_name_alt}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/wordpress-template/${taito_repo_name}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/companyname/${taito_company}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/orig-template/wordpress-template/g" 2> /dev/null {} \;
fi

# Generate ports

find . -type f -exec sed -i \
-e "s/wptemplate/${taito_project_short}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/wordpress_template/${taito_repo_name_alt}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/wordpress-template/${taito_repo_name}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/companyname/${taito_company}/g" 2> /dev/null {} \;
find . -type f -exec sed -i \
-e "s/orig-template/wordpress-template/g" 2> /dev/null {} \;

echo "Generating unique random ports (avoid conflicts with other projects)..."

ingress_port=$(shuf -i 8000-9999 -n 1)
db_port=$(shuf -i 6000-7999 -n 1)
sed ${sedi} -- "s/7587/${db_port}/g" taito-config.sh &> /dev/null
sed ${sedi} -- "s/7587/${db_port}/g" docker-compose.yaml &> /dev/null
sed ${sedi} -- "s/4635/${ingress_port}/g" docker-compose.yaml taito-config.sh \
sed -i -- "s/7587/${db_port}/g" taito-config.sh &> /dev/null
sed -i -- "s/7587/${db_port}/g" docker-compose.yaml &> /dev/null
sed -i -- "s/4635/${ingress_port}/g" docker-compose.yaml taito-config.sh \
./admin/package.json ./client/package.json &> /dev/null

echo "Replacing template variables with the given settings..."

# Replace template variables in taito-config.sh with the given settings
sed ${sedi} -- "s/export taito_company=\".*\"/export taito_company=\"${taito_company}\"/g" taito-config.sh
sed ${sedi} -- "s/export taito_family=\".*\"/export taito_family=\"${taito_family:-}\"/g" taito-config.sh
sed ${sedi} -- "s/export taito_application=\".*\"/export taito_application=\"${taito_application:-}\"/g" taito-config.sh
sed ${sedi} -- "s/export taito_suffix=\".*\"/export taito_suffix=\"${taito_suffix:-}\"/g" taito-config.sh
sed ${sedi} -- "s/export taito_repo_name=\".*\"/export taito_repo_name=\"${taito_repo_name}\"/g" taito-config.sh
sed ${sedi} -- "s/export taito_project=\".*\"/export taito_project=\"${taito_repo_name}\"/g" taito-config.sh
sed -i -- \
"s/export taito_company=\".*\"/export taito_company=\"${taito_company}\"/g" taito-config.sh
sed -i -- \
"s/export taito_family=\".*\"/export taito_family=\"${taito_family:-}\"/g" taito-config.sh
sed -i -- \
"s/export taito_application=\".*\"/export taito_application=\"${taito_application:-}\"/g" taito-config.sh
sed -i -- \
"s/export taito_suffix=\".*\"/export taito_suffix=\"${taito_suffix:-}\"/g" taito-config.sh
sed -i -- \
"s/export taito_repo_name=\".*\"/export taito_repo_name=\"${taito_repo_name}\"/g" taito-config.sh
sed -i -- \
"s/export taito_project=\".*\"/export taito_project=\"${taito_repo_name}\"/g" taito-config.sh

echo "Replacing template variables with the user specific settings..."

# Replace template variables in taito-config.sh with user specific settings
sed ${sedi} -- "s/\${template_default_organization:?}/${template_default_organization}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_domain:?}/${template_default_domain}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_zone:?}/${template_default_zone}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_zone_prod:?}/${template_default_zone_prod}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider:?}/${template_default_provider}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider:?}/${template_default_provider_billing_account:-}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider:?}/${template_default_provider_org_id}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider:?}/${template_default_provider_org_id_prod}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider_region:?}/${template_default_provider_region}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider_zone:?}/${template_default_provider_zone}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider_region_prod:?}/${template_default_provider_region_prod}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_provider_zone_prod:?}/${template_default_provider_zone_prod}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_registry:?}/${template_default_registry}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_source_git:?}/${template_default_source_git}/g" taito-config.sh
sed ${sedi} -- "s/\${template_default_dest_git:?}/${template_default_dest_git}/g" taito-config.sh
sed -i -- \
"s/\${template_default_organization:?}/${template_default_organization}/g" taito-config.sh
sed -i -- \
"s/\${template_default_domain:?}/${template_default_domain}/g" taito-config.sh
sed -i -- \
"s/\${template_default_zone:?}/${template_default_zone}/g" taito-config.sh
sed -i -- \
"s/\${template_default_zone_prod:?}/${template_default_zone_prod}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider:?}/${template_default_provider}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider_org_id:?}/${template_default_provider_org_id}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider_org_id_prod:?}/${template_default_provider_org_id_prod}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider_region:?}/${template_default_provider_region}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider_zone:?}/${template_default_provider_zone}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider_region_prod:?}/${template_default_provider_region_prod}/g" taito-config.sh
sed -i -- \
"s/\${template_default_provider_zone_prod:?}/${template_default_provider_zone_prod}/g" taito-config.sh
sed -i -- \
"s/\${template_default_registry:?}/${template_default_registry}/g" taito-config.sh
sed -i -- \
"s/\${template_default_source_git:?}/${template_default_source_git}/g" taito-config.sh
sed -i -- \
"s/\${template_default_dest_git:?}/${template_default_dest_git}/g" taito-config.sh

echo "Removing template settings from cloudbuild.yaml..."

sed ${sedi} -- "s|\${_TEMPLATE_DEFAULT_TAITO_IMAGE}|${template_default_taito_image}|g" cloudbuild.yaml
sed ${sedi} -- '/_TEMPLATE_DEFAULT_/d' cloudbuild.yaml
sed ${sedi} -- '/template_default_taito_image/d' cloudbuild.yaml
sed -i -- \
"s|\${_TEMPLATE_DEFAULT_TAITO_IMAGE}|${template_default_taito_image}|g" cloudbuild.yaml
sed -i -- '/_TEMPLATE_DEFAULT_/d' cloudbuild.yaml
sed -i -- '/template_default_taito_image/d' cloudbuild.yaml

rm -f temp
3 changes: 1 addition & 2 deletions taito-config.sh
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ export taito_storages="${taito_project}-${taito_env}"
export db_database_instance="common-mysql"
export db_database_type="mysql"
export db_database_name="${taito_project_short}${taito_env}"
export db_database_host="127.0.0.1" # TODO was localhost
export db_database_host="127.0.0.1"
export db_database_proxy_port="5001"
export db_database_port="${db_database_proxy_port}"

Expand All @@ -51,7 +51,6 @@ export dockerfile=Dockerfile
export gcloud_org_id="${template_default_provider_org_id:?}"
export gcloud_region="${template_default_provider_region:?}"
export gcloud_zone="${template_default_provider_zone:?}"
export gcloud_billing_account="${template_default_provider_billing_account:-}"
export gcloud_sql_proxy_port="${db_database_proxy_port}"
export gcloud_cdn_enabled=false

Expand Down

0 comments on commit 3b15daa

Please sign in to comment.