From 28c9e2987f748c88e8cac2a30595eff11f64a12b Mon Sep 17 00:00:00 2001 From: santhosh-tg Date: Thu, 24 Mar 2022 15:25:59 +0530 Subject: [PATCH 1/3] Update documentation: Clean up manual steps Add building blocks --- developer-docs/server-installation/index.md | 24 +++-- .../server-installation/prerequisites.md | 93 +++++++++---------- 2 files changed, 59 insertions(+), 58 deletions(-) diff --git a/developer-docs/server-installation/index.md b/developer-docs/server-installation/index.md index df064f044..c1237ea9d 100644 --- a/developer-docs/server-installation/index.md +++ b/developer-docs/server-installation/index.md @@ -19,10 +19,20 @@ This document explains the procedure to set up Sunbird on your cloud infra. This * AKS (Azure Kubernetes Service) * Ubuntu 18.04 VM's -### Infrastructure Components - -Sunbird consists of 3 major subsystems - -* Knowledge Platform also knows as Learning Platform (Taxonomy and Content) -* Data Pipeline (Data enrichment and Visualizations) -* Core Services (User, Organization, Courses, Certificates, UI and Frontend) +### Building Blocks + +Sunbird consists of these major building blocks + +* Sunbird Anuvaad +* Sunbird coKreat +* Sunbird COMET +* Sunbird cQube +* Sunbird ED +* Sunbird inQuiry +* Sunbird Knowlg +* Sunbird Lern +* Sunbird Obsrv +* Sunbird RC +* Sunbird Saral +* Sunbird UCI +* Sunbird Vakyansh diff --git a/developer-docs/server-installation/prerequisites.md b/developer-docs/server-installation/prerequisites.md index e2de864cc..dd9142a77 100644 --- a/developer-docs/server-installation/prerequisites.md +++ b/developer-docs/server-installation/prerequisites.md @@ -20,6 +20,26 @@ To successfully complete Sunbird installation, you need to have: > **Note:** Sunbird is tested only on cloud hosted Ubuntu servers in Azure. We do not provide any support for installing Sunbird on other operating systems. Sunbird might work on other cloud providers, feel free to try it out and let us know. We are actively working on achieving cloud neutrality. + +### Infra Requirements + +- Kubernetes Cluster with 4 worker nodes +- Private GitHub repository to store ansible inventory +- Fully Qualified Domain Name (FQDN) with SSL +- Azure Storage account +- Docker hub account +- Public IP +- Google Oauth Credentials +- Google V3 Recaptcha Credentials +- Maxmind city database (free or paid) +- A SMTP account (any email provider works, except Gmail for now) +- All ports must be open for internal communication (Azure Virtual Network or AWS VPC) between the VMs +- Internet and outbound access from VMs +- A sms service provider API Token (optional) +- YouTube API Token (optional) +- Slack account and slack bot with API Token (optional) + + ### Provisioning Servers Before you start the installation process, ensure that you have the required infrastructure mentioned below. It is the bare minimum for a full fledged Sunbird setup. Every component in Sunbird can scale horizontally / vertically by adding additional resources. @@ -45,67 +65,38 @@ Easiest way to use the script will be to use azure cloud shell, as the cloud she If you want to run this on your local machine, Follow this [guide](https://docs.microsoft.com/en-us/azure/developer/ansible/install-on-linux-vm?tabs=azure-cli#install-ansible-on-the-virtual-machine). -``` -git clone https://github.com/project-sunbird/sunbird-devops -b release-4.3.0 -cd sunbird-devops/deploy -# Update the necessary variables in playbook -ansible-playbook -c local azure-provision.yaml -# Resulting infrastructure infromation will be stored in sunbird-devops/deploy/azure-resources.txt file. -``` -### Infra Requirements +#### Create service principal for kubernetes cluster -- Kubernetes Cluster with 4 worker nodes -- Private GitHub repository to store ansible inventory -- Fully Qualified Domain Name (FQDN) with SSL -- Azure Storage account -- Docker hub account -- Public IP -- Google Oauth Credentials -- Google V3 Recaptcha Credentials -- Maxmind city database (free or paid) -- A SMTP account (any email provider works, except Gmail for now) -- All ports must be open for internal communication (Azure Virtual Network or AWS VPC) between the VMs -- Internet and outbound access from VMs -- A sms service provider API Token (optional) -- YouTube API Token (optional) -- Slack account and slack bot with API Token (optional) +Command to create SP: +``` +az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/" --name NameOfSP -### Creating the AKS cluster +# Note down the output of above command, we will need those to fill-up some variables in the ansible playbook +``` -> **Note** Follow the steps given below to create the Kubernetes cluster in Azure. For other clouds, please visit the respective cloud provider website -The AKS cluster and VM's should be in same vnet. If they are in different vnet, you have to peer the vnets. To successfully peer, the IP address of the vnets should not overlap. -- Create a service principal and assign contributor role to service principal -- Get the secrets and client id of service principal -- Click [here](https://docs.microsoft.com/en-us/cli/azure/create-an-azure-service-principal-azure-cli) for more details -- Create the AKS cluster either via Azure portal or using `az aks` command line -- Refer to Azure documentation for all the available options -- Below is a sample command which you can use - +#### Clone the repo and go to deploy directory - ```bash -az aks create --resource-group --node-resource-group --name --node-count 4 --admin-username deployer --kubernetes-version 1.19.9 --service-principal "" --node-vm-size Standard_D4s_v3 --client-secret "" --network-plugin azure --ssh-key-value @deployer.pub -l --vm-set-type VirtualMachineScaleSets --vnet-subnet-id /subscriptions//resourceGroups//providers/Microsoft.Network/virtualNetworks//subnets/ ``` +git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0 -> Note: Ensure you have allocated at least 1024 IP's for your Kubernetes subnet (CIDR notation as x.x.x.x/22) - -Get the kubeconfig file for your cluster with the below command - - -```bash -az aks get-credentials --resource-group --name --file k8s.yaml +cd sunbird-devops/deploy ``` -### Configuring the Azure storage account -- Update the CORS rule for the storage account as follows: +#### Update variables in azure-provision.yaml file - ```bash - Allowed Origins: * - Allowed Methods: GET,HEAD,OPTIONS, PUT - Allowed Headers: Access-Control-Allow-Origin,Access-Control-Allow-Method,Origin,x-ms-meta-qq,x-ms-blob-type,x-ms-blob-content-type,Content-Type - Exposed Headers: Access-Control-Allow-Origin,Access-Control-Allow-Methods - Max Age: 200 +``` +version: "ChangeThis" # sunbird version Ex: 4.7.0 +sp_client_id: "ChangeThis" # substitute the value of `appId` from the output of the create SP command +sp_client_secret: "ChangeThis" # substitute the value of `password` from the output of the create SP command +ssh_public_key_file_location: "ChangeThis" # path to your ssh public key Ex: /home/sunbird/.ssh/my_pub.key +``` + +#### Run the playbook +``` +ansible-playbook -c local azure-provision.yaml - ``` - -- Disable **Secure transfer required** in storage account configuration \ No newline at end of file +# Resulting infrastructure infromation will be stored in sunbird-devops/deploy/azure-resources.txt file. +``` From ff672cd14da65dbc877b0c16b615d70b30cefbcd Mon Sep 17 00:00:00 2001 From: santhosh-tg Date: Mon, 28 Mar 2022 17:43:43 +0530 Subject: [PATCH 2/3] Update 4.7.0 documentation --- _data/sidebar.yaml | 4 +- .../server-installation/build-and-deploy.md | 49 +------------- developer-docs/server-installation/index.md | 7 -- .../server-installation/infra-setup.md | 65 +++++++++++++++++++ .../server-installation/jenkins-setup.md | 11 ++-- .../server-installation/prerequisites.md | 47 -------------- 6 files changed, 73 insertions(+), 110 deletions(-) create mode 100644 developer-docs/server-installation/infra-setup.md diff --git a/_data/sidebar.yaml b/_data/sidebar.yaml index ba00dd3e3..c4e7773f3 100644 --- a/_data/sidebar.yaml +++ b/_data/sidebar.yaml @@ -7,8 +7,8 @@ subfolderitems: - page: Pre-Requisites url: developer-docs/server-installation/prerequisites/ - - page: Optional Load Balancer Setup - url: developer-docs/server-installation/load-balancer-setup/ + - page: Infra Setup + url: developer-docs/server-installation/infra-setup/ - page: Ansible Variables url: developer-docs/server-installation/ansible-variables/ - page: Jenkins Setup diff --git a/developer-docs/server-installation/build-and-deploy.md b/developer-docs/server-installation/build-and-deploy.md index 2532c5c30..ff59398df 100644 --- a/developer-docs/server-installation/build-and-deploy.md +++ b/developer-docs/server-installation/build-and-deploy.md @@ -10,56 +10,9 @@ allowSearch: true ## Overview -To install Sunbird `release-4.6.0`, we will need to run many Jenkins jobs and a couple of manual steps. Some of the jobs need to be run in a specific order. This page details out the list of jobs, their order and the github release tags to be used for each job. +To install Sunbird `release-4.7.0`, we will need to run many Jenkins jobs and a couple of manual steps. Some of the jobs need to be run in a specific order. This page details out the list of jobs, their order and the github release tags to be used for each job. -### Installing Sunbird - -#### Create Plugin Containers / Folder Structure - -- We will create few folders in azure for storing sunbird plugins -- Our deployment job always tries to delete the folder first before uploading, so we need to have the placeholders before running the jobs -- We will create the folders by uploading random files from a random folder on your PC -- Create a public container with the name you have set for the variable `sunbird_content_azure_storage_container` and run the below commands - -```bash -az storage blob upload-batch --destination sunbird_content_azure_storage_container/collection-editor --source some_folder --account-name storage_account_name --account-key storage_account_key - -az storage blob upload-batch --destination sunbird_content_azure_storage_container/generic-editor --source some_folder --account-name storage_account_name --account-key storage_account_key - -az storage blob upload-batch --destination sunbird_content_azure_storage_container/content-editor --source some_folder --account-name storage_account_name --account-key storage_account_key - -az storage blob upload-batch --destination sunbird_content_azure_storage_container/v3/preview --source some_folder --account-name storage_account_name --account-key storage_account_key -``` - -- Upload the default T&C file to `sunbird_content_azure_storage_container` container in a folder named **terms-and-conditions**. You can get a copy of the sample T&C file from [here](https://sunbirdpublic.blob.core.windows.net/installation/terms-and-conditions/terms-and-conditions-v9.html). You are free to edit the HTML as per your requirements -- Create a private container named `label` in Azure -- Run the below command in your Jenkins VM where you cloned the sunbird-devops repo - -```bash -cd sunbird-devops/utils/portal -az storage blob upload-batch --destination label --source labels --account-name storage_account_name --account-key storage_account_key -``` - -#### Upload Initial Plugins - -- Upload the initial set of plugins to `sunbird_content_azure_storage_container` container -- Download the initial Plugins from [here](https://sunbirdpublic.blob.core.windows.net/installation/content-plugins.zip) -- Run the below command from the directory where **content-plugins.zip** is unzipped - -```bash -az storage blob upload-batch --destination sunbird_content_azure_storage_container/content-plugins --source content-plugins --account-name storage_account_name --account-key storage_account_key -``` - -- Create a container named `public` in your Azure storage account from the Azure portal with public access level -- Create a container named `artifacts` in your Azure storage account from the Azure portal with private access level - - -#### Upload Maxmind Database to Azure Blob - -- Download the Maxmind city database in zip format from Maxmind website -- Upload the zip file to the `artifacts` container in Azure - #### Code Builds > **Note**: diff --git a/developer-docs/server-installation/index.md b/developer-docs/server-installation/index.md index c1237ea9d..c0ec5ffa0 100644 --- a/developer-docs/server-installation/index.md +++ b/developer-docs/server-installation/index.md @@ -23,16 +23,9 @@ This document explains the procedure to set up Sunbird on your cloud infra. This Sunbird consists of these major building blocks -* Sunbird Anuvaad -* Sunbird coKreat -* Sunbird COMET -* Sunbird cQube -* Sunbird ED * Sunbird inQuiry * Sunbird Knowlg * Sunbird Lern * Sunbird Obsrv * Sunbird RC -* Sunbird Saral * Sunbird UCI -* Sunbird Vakyansh diff --git a/developer-docs/server-installation/infra-setup.md b/developer-docs/server-installation/infra-setup.md new file mode 100644 index 000000000..5411d2dc6 --- /dev/null +++ b/developer-docs/server-installation/infra-setup.md @@ -0,0 +1,65 @@ +#### Automation on Azure infrastructure creation + +You can run the following steps to create azure infrastructure using ansible. + +Easiest way to use the script will be to use azure cloud shell, as the cloud shell comes with all prerequisites bundled. +- login to portal.azure.com +- click on the cloudshell -> select bash ( if you're using it for the first time ) + +If you want to run this on your local machine, Follow this [guide](https://docs.microsoft.com/en-us/azure/developer/ansible/install-on-linux-vm?tabs=azure-cli#install-ansible-on-the-virtual-machine). + + +#### Create service principal for kubernetes cluster + +Command to create SP: +```bash +az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/" --name + +# Replace the <> with your values +``` + +Note down the output of above command as we will need those to fill-up some variables in the ansible playbook + +#### Create Maxmind licence key +```bash +1. Signup at maxmind.com if you don't have an account +2. Goto https://www.maxmind.com/en/accounts/current/license-key +3. Click Generate a new license key. +``` + +#### Clone the repo and go to deploy directory + +```bash +git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0 + +cd sunbird-devops/deploy +``` + +#### Update variables in azure-provision.yaml file + +```bash +version: "ChangeThis" # sunbird version Ex: 4.7.0 +sp_client_id: "ChangeThis" # substitute the value of `appId` from the output of the create SP command +sp_client_secret: "ChangeThis" # substitute the value of `password` from the output of the create SP command +ssh_public_key_file_location: "ChangeThis" # path to your ssh public key Ex: /home/sunbird/.ssh/my_pub.key +geoip_licence_key: "ChangeThis" # substitute the one that you generated at # Create Maxmind licence key step +``` + +#### Run the playbook +```bash +ansible-playbook -c local azure-provision.yaml + +# Resulting infrastructure information will be stored in sunbird-devops/deploy/azure-resources.txt file. +``` + +Running the playbook creates these azure resources + +* Resource group +* VNET and subnet +* 6 VMs (size: Standard_D4s_v4) +* Public IP for Jenkins +* AKS cluster with 4 nodes (size: Standard_D4s_v4) +* Storage account +* ACR ( Image registry) + +and performs some steps needed for the sunbird installation like uploading maxmind database to blob storage etc. Please refer to the task section of the playbook to get the complete list diff --git a/developer-docs/server-installation/jenkins-setup.md b/developer-docs/server-installation/jenkins-setup.md index e8012d058..45a09f206 100644 --- a/developer-docs/server-installation/jenkins-setup.md +++ b/developer-docs/server-installation/jenkins-setup.md @@ -14,9 +14,8 @@ Jenkins is used to build, deploy and setup the infrastructure for Sunbird. Almos - SSH to the Jenkins server and enter the following commands - ```bash - git clone https://github.com/project-sunbird/sunbird-devops.git - cd sunbird-devops && git checkout tags/release-4.3.0 -b release-4.3.0 - cd deploy/jenkins + git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0 + cd sunbird-devops/deploy/jenkins sudo bash jenkins-server-setup.sh ``` @@ -59,9 +58,9 @@ Jenkins is used to build, deploy and setup the infrastructure for Sunbird. Almos ``` sudo su jenkins curl -SsL -o ~/jenkins.yaml https://raw.githubusercontent.com/project-sunbird/sunbird-devops/release-4.3.0/deploy/jenkins/jenkins.yaml - + # Replace all placeholders ${VALUE} with proper values - # For example, + # For example, # username: "${GH_USERNAME}" to username: "mygithubusername" vim ~/jenkins.yaml ``` @@ -87,4 +86,4 @@ Jenkins is used to build, deploy and setup the infrastructure for Sunbird. Almos jenkins ALL=(ALL) NOPASSWD:ALL ``` -- Reboot the Jenkins VM (`sudo reboot`) \ No newline at end of file +- Reboot the Jenkins VM (`sudo reboot`) diff --git a/developer-docs/server-installation/prerequisites.md b/developer-docs/server-installation/prerequisites.md index dd9142a77..8b3b8aca7 100644 --- a/developer-docs/server-installation/prerequisites.md +++ b/developer-docs/server-installation/prerequisites.md @@ -53,50 +53,3 @@ Before you start the installation process, ensure that you have the required inf | Yarn | 4core 16G 60G HDD | 1 | | Other Services | 4core 16G 60G HDD | 1 | | Basic Load Balancers | - | 2 (Optional) | - - -#### Automation on Azure infrastructure creation - -You can run the following steps to create azure infrastructure using ansible. - -Easiest way to use the script will be to use azure cloud shell, as the cloud shell comes with all prerequisites bundled. -- login to portal.azure.com -- click on the cloudshell -> select bash ( if you're using it for the first time ) - -If you want to run this on your local machine, Follow this [guide](https://docs.microsoft.com/en-us/azure/developer/ansible/install-on-linux-vm?tabs=azure-cli#install-ansible-on-the-virtual-machine). - - -#### Create service principal for kubernetes cluster - -Command to create SP: -``` -az ad sp create-for-rbac --role="Contributor" --scopes="/subscriptions/" --name NameOfSP - -# Note down the output of above command, we will need those to fill-up some variables in the ansible playbook -``` - - -#### Clone the repo and go to deploy directory - -``` -git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0 - -cd sunbird-devops/deploy -``` - - -#### Update variables in azure-provision.yaml file - -``` -version: "ChangeThis" # sunbird version Ex: 4.7.0 -sp_client_id: "ChangeThis" # substitute the value of `appId` from the output of the create SP command -sp_client_secret: "ChangeThis" # substitute the value of `password` from the output of the create SP command -ssh_public_key_file_location: "ChangeThis" # path to your ssh public key Ex: /home/sunbird/.ssh/my_pub.key -``` - -#### Run the playbook -``` -ansible-playbook -c local azure-provision.yaml - -# Resulting infrastructure infromation will be stored in sunbird-devops/deploy/azure-resources.txt file. -``` From 2c24d44f291a3db40345f44e7815be033f4eb9dc Mon Sep 17 00:00:00 2001 From: santhosh-tg Date: Mon, 28 Mar 2022 19:04:16 +0530 Subject: [PATCH 3/3] Update 4.7.0 documentation --- .../server-installation/ansible-variables.md | 8 +++----- developer-docs/server-installation/infra-setup.md | 11 +++++++++++ 2 files changed, 14 insertions(+), 5 deletions(-) diff --git a/developer-docs/server-installation/ansible-variables.md b/developer-docs/server-installation/ansible-variables.md index 31fa8f615..2d068b56d 100644 --- a/developer-docs/server-installation/ansible-variables.md +++ b/developer-docs/server-installation/ansible-variables.md @@ -14,16 +14,14 @@ Ansible is the configuration management system used in Sunbird. The infrastructu Use the following git commands sequentially to clone and update your private GitHub repository - ```bash -git clone https://github.com/project-sunbird/sunbird-devops -cd sunbird-devops -git checkout tags/release-4.3.0 -b release-4.3.0 + git clone https://github.com/project-sunbird/sunbird-devops -b release-4.7.0 ``` - Copy the directory `sunbird-devops/private_repo/ansible` to your private repo - Update the files **common.yml**, **hosts**, and **secrets.yml** under **Core**, **KnowledgePlatform** and **DataPipeline** directories. After updating, push them to your private repo branch - Your private repo structure starting from the root path should be exactly as shown below - + ```bash ansible └── inventory @@ -137,4 +135,4 @@ ansible log-es-1, log-es-backup, log-es - \ No newline at end of file + diff --git a/developer-docs/server-installation/infra-setup.md b/developer-docs/server-installation/infra-setup.md index 5411d2dc6..2cc7c6d7f 100644 --- a/developer-docs/server-installation/infra-setup.md +++ b/developer-docs/server-installation/infra-setup.md @@ -1,3 +1,14 @@ +--- +title: Infra Setup +page_title: Infra Setup +description: Explains the infra setup and automation +keywords: infra, setup, automation, install, sunbird +allowSearch: true + +--- + + + #### Automation on Azure infrastructure creation You can run the following steps to create azure infrastructure using ansible.