Merge branch 'ash' into 'master'

Merge ash release branch to master branch.

Closes VIYAARK-64 and VIYAARK-46

See merge request sassoftware/viya-ark!283

CHANGELOG.md

@@ -1,6 +1,25 @@
 # Changelog for SAS Viya ARK
 
 <!-- LATEST RELEASE START -->
+## Viya35-ark-1.1 - December 19, 2019
+
+- **Summary**:
+  Enhancements and fixes related to Upgrade, Deployment Report, Pre-Installation playbooks and documentation.
+- Issues addressed:
+  - General
+    - VIYAARK-64 - Known Issues section added to main README
+  - SAS Viya Upgrade Tasks Playbooks
+    - VIYAARK-45 - DEFAULT_BACKUP_SCHEDULE job not recreated after upgrade from SAS Viya 3.3 to SAS Viya 3.4/3.5
+    - VIYAARK-41 - Pre-Upgrade playbook fails if SAS Viya 3.5 software order adds [ModelServer]
+  - SAS Viya Deployment Report
+    - VIYAARK-46 - Temporarily disable hotfix report for earlier release
+  - SAS Viya Pre-Installation Playbook
+    - VIYAARK-43 - Remove libXp.i686 from required software packages
+    - VIYAARK-25 - Non-supported Java version check done in normal mode as well as check mode
+    - VIYAARK-50 - Kernel semaphore performance configuration task not adjusting settings as expected (Python 2) and failing playbook (Python 3)
+
+<!-- LATEST RELEASE END -->
+
 ## Viya35-ark-1.0 - November 19, 2019
 
 - **Summary**:
@@ -34,8 +53,6 @@
     - DEPENBDAT-627 - Ansible 2.9 support for viya-upgrade related playbook.
 - Ansible Support: Ansible 2.7.2 - Ansible 2.9
 
-<!-- LATEST RELEASE END -->
-
 ## Viya34-ark-1.7 - August 22, 2019
 
 - **Summary**:

README.md

@@ -34,6 +34,11 @@ Tool support for the latest release of SAS Viya:
 Related Tools Not Contained in SAS Viya ARK:
 * SAS Viya ARK only supports Linux.  For Windows supporting tools, go to [SAS Viya Deployment Assistant for Windows](https://support.sas.com/en/documentation/install-center/viya/deployment-tools/34/deployment-assistant-windows.html).
 
+## Known Support Issues
+The following issues are known and may impact the expected usage or performance of the Viya ARK tools.
+
+* SAS Viya 3.5 supports Linux on Power on a limited availability basis. The SAS Viya ARK tools do not support Linux on Power in the Viya35-ark-1.0 release.
+
 ## License
 
 This project is licensed under the [Apache 2.0 License](LICENSE).

playbooks/deployment-report/README.md

@@ -2,15 +2,8 @@
 
 ## Introduction
 This playbook gathers information about SAS Viya software and the hosts where it is deployed.
-Furthermore, it optionally compares the packages installed on the system with the most up to 
-date versions to determine which hotfixes have not been applied.  The data gathered is then 
-written to disk as a YAML-formatted data file and a static web-page for easily viewing the data. 
-
-Hotfix information in the report include:
-* The hotfixes not applied
-* The release date of each hotfix
-* Each package and version associated with the hotfix
-* The number and title of each SASNote associated with the hotfix
+The data gathered is then written to disk as a YAML-formatted data file and a static web-page 
+for easily viewing the data. 
 
 Deployment attributes in the report include:
 * Operating system information and architecture
@@ -32,7 +25,6 @@ The output files written to the `sas_viya_playbook/` are:
 * Install SAS Viya software using the SAS-provided `sas_viya_playbook`.
 * Obtain a local copy of the inventory file used when deploying the SAS Viya software.
 * Verify that the user has sudoers privileges.
-* Connectivity to sas.com (for the Hotfix Report Only)
 
 ## Running the Playbook
 To run the playbook, execute the following command:
@@ -59,20 +51,7 @@ To exclude the static web page and only create the report data:
   ansible-playbook viya-ark/playbooks/deployment-report/viya-deployment-report.yml -e "exclude_html=true"
   ```
 
-To exclude the hotfix report:
-  ```bash
-  ansible-playbook viya-ark/playbooks/deployment-report/viya-deployment-report.yml -e "include_hotfix_report=False"
-  ```
-
 To force the creation of the report files into the current directory:
   ```bash
   ansible-playbook viya-ark/playbooks/deployment-report/viya-deployment-report.yml -e 'output_dir=./'
   ```
-To specify an alternate location for the published hotfix data:
-  ```bash
-  ansible-playbook viya-ark/playbooks/deployment-report/viya-deployment-report.yml -e 'hotfix_url=<URL_To_Hotfix_Master_File>'
-  ```
-To specify an alternate hotfix file:
-  ```bash
-  ansible-playbook viya-ark/playbooks/deployment-report/viya-deployment-report.yml -e 'hotfix__master_file=<Master_File_Name>'
-  ```

playbooks/deployment-report/library/process_sas_host_details.py

@@ -41,22 +41,24 @@
         description:
             - The name of the dict object registered with hostvars which will contain the host details.
         required: true
-    include_hotfix_report:
-        description:
-            - Whether or not to include the hotfix report.
-        required:  false
-        default:   true
-    hotfix_url:
-        description:
-            -  The URL to look for the published hotfixes.
-        require:  False
-        default:  http://ftp.sas.com/techsup/download/hotfix/HF2/util01/Viya/data/
-    hotfix_master_file:
-        description:
-            -  The file that holds all of the hotfix infomration.
-        require:  False
-        default:  Viya_Update_Page_Index.xml
 '''
+# The following are options for the hotfix report, which has been temporarily disabled for this release.
+#    include_hotfix_report:
+#        description:
+#            - Whether or not to include the hotfix report.
+#        required:  false
+#        default:   true
+#    hotfix_url:
+#        description:
+#            -  The URL to look for the published hotfixes.
+#        require:  False
+#        default:  http://ftp.sas.com/techsup/download/hotfix/HF2/util01/Viya/data/
+#    hotfix_master_file:
+#        description:
+#            -  The file that holds all of the hotfix infomration.
+#        require:  False
+#        default:  Viya_Update_Page_Index.xml
+#'''
 
 EXAMPLES = '''
 # Process SAS deployment information

playbooks/deployment-report/viya-deployment-report.yml

@@ -30,7 +30,8 @@
     report_file_name: "viya_deployment_report"
     report_data_file_name: "{{ report_file_name + '_data' }}"
     output_dir: "{{ playbook_dir + '/../../..' }}"
-    include_hotfix_report: True
+    # The hotfix report has been temporarily disabled this release.
+    include_hotfix_report: False
     hotfix_url: "http://ftp.sas.com/techsup/download/hotfix/HF2/util01/Viya/data/"
     hotfix_master_file: "Viya_Update_Page_Index.xml"
 

playbooks/pre-install-playbook/README.md

@@ -65,9 +65,7 @@ You can elicit the SAS Viya 3.4 behavior by passing in extra parameter ```-e viy
 ### Supported Java Runtime Environment (JRE) Versions
 Due to some compatibility issues with some components of SAS Viya 3.5, a minimum version of Java is now required.  That minimum version can be found by examining the  ```min_java_version``` variable in roles/viya-ark.preinstall/defaults/main.yml.
 
-The tasks controlled by the ```required_packages_config``` tag will have different behavior in SAS Viya 3.5.  If there is a previously installed Java and that version is less than the minimum version required for SAS Viya 3.5, the playbook will fail if executed with ```--check```.  If it is executed in normal mode, the playbook will behave as it did with SAS Viya 3.4 and, if no Java is installed, the latest Java will be installed.  The "latest" Java is determined by the value of ```java_openjdk_version``` found in roles/viya-ark.preinstall/vars/default.yml (or OS-specific var files in that same directory).  The actual version of Java that is installed is therefore dependent on what version is available in the repositories set up for the environment.
-If Java is already installed, and the playbook is executed in normal mode, no update to the Java package will occur.   The version found to be installed will be reflected in a debug message for awareness sake.  It is recommended to execute ```--check``` mode in order to validate any previously installed Java version with respect
-to the SAS Viya 3.5 minimum supported Java version requirement.
+The tasks controlled by the ```required_packages_config``` tag will have different behavior in SAS Viya 3.5.  If there is a previously installed Java and that version is less than the minimum version required for SAS Viya 3.5, the playbook will fail and report the version mismatch.  If no Java is installed, the latest available Java will be installed and the minimum version check tasks will be executed.  The "latest" Java is determined by the value of ```java_openjdk_version``` found in roles/viya-ark.preinstall/vars/(OS-specific).yml.  The actual version of Java that is installed is therefore dependent on what version is available in the repositories set up for the environment.
 
 ### Required package version for RHEL deployments
 Starting with SAS Viya 3.5, minimum versions of certain packages are now required for standard RHEL deployments; please note that this does NOT apply to deployments on Oracle RHEL.  The required packages and minimum versions can be found by examining the  ```required_package_versions_rhel``` variable in roles/viya-ark.preinstall/defaults/main.yml. These packages must either be already installed at the minimum version specified, or available in a registered repository at deployment time. The pre-install playbook will fail execution if neither of these conditions are met.

playbooks/pre-install-playbook/roles/viya-ark.preinstall/defaults/main.yml

@@ -7,8 +7,6 @@ os_major_version_1: 7
 os_major_minor_version_1: 7.1
 os_major_version_2: 6
 os_major_minor_version_2: 6.7
-os_major_version_3: 8
-os_major_minor_version_3: 8.0
 suse_os_major_version: 12
 suse_os_major_minor_version: 12.2
 
@@ -135,7 +133,6 @@ packages_general:
   - "{{ java_openjdk_version }}"
   - numactl
   - "{{ libXp_x86_64_pkg }}"
-  - "{{ libXp_i686_pkg }}"
   - "{{ libXext_pkg }}"
   - "{{ libXmu_pkg }}"
   - "{{ libXtst_pkg }}"

playbooks/pre-install-playbook/roles/viya-ark.preinstall/tasks/pre.os_version_check.yml

@@ -18,7 +18,7 @@
         - ansible_os_family == redhat_os_name or ansible_os_family == suse_os_name 
       msg: |
         The OS of this server is not supported.
-        Viya requires {{redhat_os_name}} {{os_major_version_1}}, {{os_major_version_2}} or {{os_major_version_3}}, (at least {{os_major_minor_version_1}}, {{os_major_minor_version_2}} or {{os_major_minor_version_3}} ) or {{suse_os_name}} at least version {{suse_os_major_minor_version}}
+        Viya requires {{redhat_os_name}} {{os_major_version_1}}, {{os_major_version_2}}, (at least {{os_major_minor_version_1}}, {{os_major_minor_version_2}} ) or {{suse_os_name}} at least version {{suse_os_major_minor_version}}
         You seem to have {{ansible_os_family}} {{ansible_distribution_version}}
 
   - name: Ensure that the Operating System version is supported when on RedHat (this includes Oracle Linux)
@@ -28,23 +28,17 @@
         - ansible_distribution_major_version is version(os_major_version_1, '==')
           or
           ansible_distribution_major_version is version(os_major_version_2, '==')
-          or
-          ansible_distribution_major_version is version(os_major_version_3, '==')
-        # at least 6.7 if 6 or at least 7.1 if 7 or at least 8.0 if 8
+        # at least 6.7 if 6 or at least 7.1 if 7
         - (ansible_distribution_major_version is version(os_major_version_1, '==')
           and
           ansible_distribution_version is version(os_major_minor_version_1, '>='))
           or
           (ansible_distribution_major_version is version(os_major_version_2, '==')
           and
           ansible_distribution_version is version(os_major_minor_version_2, '>='))
-          or 
-          (ansible_distribution_major_version is version(os_major_version_3, '==')
-          and
-          ansible_distribution_version is version(os_major_minor_version_3, '>='))
       msg: |
         The OS of this server is not supported.
-        Viya requires {{redhat_os_name}} {{os_major_version_1}}, {{os_major_version_2}} or {{os_major_version_3}}, (at least {{os_major_minor_version_1}}, {{os_major_minor_version_2}} or {{os_major_minor_version_3}} )
+        Viya requires {{redhat_os_name}} {{os_major_version_1}}, {{os_major_version_2}}, (at least {{os_major_minor_version_1}}, {{os_major_minor_version_2}} )
         You seem to have {{ansible_os_family}} {{ansible_distribution_version}}
     when: ansible_os_family == redhat_os_name
 

playbooks/pre-install-playbook/roles/viya-ark.preinstall/tasks/pre.performance_config.yml

@@ -32,7 +32,7 @@
         kernel_SEMMSL: "{{ item.value }}"
     when:
       - item.name is match("kernel.sem.SEMMSL")
-      - item.value > kernel_SEMMSL
+      - item.value > kernel_SEMMSL|int
     with_items:
       - "{{ semaphores }}"
 
@@ -41,7 +41,7 @@
         kernel_SEMMNS: "{{ item.value }}"
     when:
       - item.name is match("kernel.sem.SEMMNS")
-      - item.value > kernel_SEMMNS
+      - item.value > kernel_SEMMNS|int
     with_items:
       - "{{ semaphores }}"
 
@@ -50,7 +50,7 @@
         kernel_SEMOPM: "{{ item.value }}"
     when:
       - item.name is match("kernel.sem.SEMOPM")
-      - item.value > kernel_SEMOPM
+      - item.value > kernel_SEMOPM|int
     with_items:
       - "{{ semaphores }}"
 
@@ -59,7 +59,7 @@
         kernel_SEMMNI: "{{ item.value }}"
     when:
       - item.name is match("kernel.sem.SEMMNI")
-      - item.value > kernel_SEMMNI
+      - item.value > kernel_SEMMNI|int
     with_items:
       - "{{ semaphores }}"
 

playbooks/pre-install-playbook/roles/viya-ark.preinstall/tasks/pre.required_packages_config.yml

@@ -17,9 +17,29 @@
     when: ((ansible_distribution == redhat_os_name|string or ansible_distribution == oracle_os_name|string) and viya_version is version('3.5','>='))
     tags:
       - packages
+      - package_version_check
+
+
+  # Note that Oracle 7 distributions reflect being RedHat ansible_os_family
+  - name: Include OS Family specific package names
+    include_vars: "{{ item }}"
+    with_first_found:
+      - "{{ ansible_os_family }}.yml"
+      - default.yml
+
+
+  ## works for all OS Families
+  - name: Ensures required packages are present
+    package:
+      name: "{{ item }}"
+      state: present
+    with_items:
+      - "{{packages_general}}"
+    tags:
+      - packages
 
   - block:
-    ## block start determining supported version of java opensdk
+    ## block start determining supported version of java openjdk
     - name: Get what version of Java might be installed
       shell: java -version 2>&1 | grep version | awk '{print $3}' | sed 's/"//g'
       register: java_version
@@ -43,44 +63,11 @@
 
     # exec during check mode
     check_mode: no
-    # only exec the block when executing in check mode
-    when: ansible_check_mode |bool and viya_version is version('3.5','>=')
+    when: viya_version is version('3.5','>=')
     # end block
     tags:    
       - jdk_check
 
-
-  # Note that Oracle 7 distributions reflect being RedHat ansible_os_family
-  - name: Include OS Family specific package names
-    include_vars: "{{ item }}"
-    with_first_found:
-      - "{{ ansible_os_family }}.yml"
-      - default.yml
-
-  ## works for all OS Families
-  - name: Ensures required packages are present
-    package:
-      name: "{{ item }}"
-      state: present
-    with_items:
-      - "{{packages_general}}"
-    tags:
-      - packages
-
-  # Get and Show the version of Java installed.  A help to debug java version issues.
-  - name: Get the installed version of Java
-    shell: java -version 2>&1 | grep version | awk '{print $3}' | sed 's/"//g'
-    register: java_version_present
-    check_mode: no
-    tags:
-      - packages
-
-  - name: Show the installed version of Java
-    debug: var=java_version_present.stdout
-    tags:
-      - packages
-
-
   ## RHEL 6 specific
   - name: Ensures required RHEL6 packages are present
     package:

playbooks/pre-install-playbook/roles/viya-ark.preinstall/vars/RedHat.yml

@@ -2,7 +2,6 @@
 # OS specific package names for RedHat and OracleLinux
 java_openjdk_version: java-1.8.0-openjdk
 libXp_x86_64_pkg: libXp.x86_64
-libXp_i686_pkg: libXp.i686
 libXext_pkg: libXext
 libXmu_pkg: libXmu
 libXtst_pkg: libXtst

playbooks/pre-install-playbook/roles/viya-ark.preinstall/vars/Suse.yml

@@ -2,7 +2,6 @@
 # OS specific package names for SuSE/SLES
 java_openjdk_version: java-1_8_0-openjdk
 libXp_x86_64_pkg: libXpm4
-libXp_i686_pkg: libXpm4
 libXext_pkg: libXext6
 libXmu_pkg: libXmu6
 libXtst_pkg: libXtst6

playbooks/pre-install-playbook/roles/viya-ark.preinstall/vars/default.yml

@@ -2,7 +2,6 @@
 # OS specific package names for RedHat
 java_openjdk_version: java-1.8.0-openjdk
 libXp_x86_64_pkg: libXp.x86_64
-libXp_i686_pkg: libXp.i686
 libXext_pkg: libXext
 libXmu_pkg: libXmu
 libXtst_pkg: libXtst