From 42a80718af53caa7e115769c21fd3c39c245185c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Andr=C3=A9=20Marcel=20Guti=C3=A9rrez=20Ben=C3=ADtez?= <68956970+AndreMarcel99@users.noreply.github.com> Date: Mon, 13 Jan 2025 11:01:13 -0600 Subject: [PATCH] [Module][1717]new_module_zfs_resize (#1767) * Remove Hardcoded hlq * Fix removal * Add some solution * First version module * Get size correctly * Refactor module and add new module_utils * Add comments * Fix documentation * Less code, working with verbose * Fix comments * Fix comments and verbose * Fix comments * Fix documentation * Fix trc and cyl * Generate file * Fix names and operations * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * hange to space_type * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Change names * Add comments * Return size of space_type given by the user * Add obj as name * Add first two basic test cases * Fix documentation * Fix documentation * Fix documentation * Fix documentation * Fix comments and add trace_path * Fix verbose * Add validation to full traceback * Fix error message * Add fix type * Add test case * Remove cp * Fix dir_path * Add fragment * Fix tracks and cylinder * Add test case of space_type * Add test cases for all positives cases * Update Github bug and enhancement templates with latest versions (#1820) * Updated bug issue * Updated collaboration issue * Updated doc issue * Update plugins/modules/zos_fetch.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_fetch.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_fetch.py Co-authored-by: Fernando Flores * Fix execpton handle * Fix sanity * Adjust best practices * Add negative correct error * Fix sanity * Use raises * Update mounts.env (#1826) Swapped old zoau v1.1.0 to use as zoau 1.3.4 ga build * [zos_mvs_raw] Find VSAM cluster when DISP=OLD and find data and index resource types (#1822) * Added fix for zos_find * Added tests for fix * Updated changelog * Fix error messages and more * Fix test cases issues positives and add negatives * Fix resize call * End portability * Add fragment * [enabler][zos_copy]Add_error_message (#1821) * Add enhance error message * Add fragment * Update plugins/modules/zos_copy.py Co-authored-by: Fernando Flores * Update 1821-Add_error_message.yml * Update zos_copy.py --------- Co-authored-by: Fernando Flores * Get aggregation size available for everyone * Fix no change on type * Add validation * Add extra asserts * Enhance error message * Add space * Updates to release notes and support for ansible-core 2.18 venv for AC tool (#1829) * Update minimum version of z/OS for the GA of 1.12 Signed-off-by: ddimatos * Update AAP link Signed-off-by: ddimatos * Venv for 2.18 Signed-off-by: ddimatos * remove merge flags Signed-off-by: ddimatos * Merged release into dev * fixed test case * Fixed mvs raw test --------- Signed-off-by: ddimatos Co-authored-by: Fernando Flores * [Enabler] [AC tool] Add more options when testing with the AC tool (#1827) * Change flag used to run specific test * Add more env vars needed to run tests * Add verbose option to ac_test * Add mark and stop options to ac_test * Refactor pytest call Also changes how verbosity is handled, as well as adds printing of each pytest command run. * Update config to support new way of handling env vars in tests * Add profile option to ac_lint * Add support to run multiple files in ac_test * Change boolean options to flags Options `debug` (ac_test, test_concurrent) and `stop` (ac_test) can be now used simply as flags, without the need to add a `"true"` string after the option. * Document and clean up arg parser * Update more boolean options to be flags * Refactor pytest call * Add option 'volumes' to ac_test * Add user option and tweak verbosity * Add ac-test-required command * Add changelog fragment * Fix trace dataset * Add trace * Add space * [Enabler] [module_utils/copy.py] Replace calls to cp for dcp's Python API (#1831) * Replace calls to cp for dcp * Replace copy functions in copy.py * Replace copy calls in modules * Fix typo * Fix return statement * Add changelog fragment * Fix sanity issues * Add test cases * Add test case to validate test lineinfile * Add fragment * Update tests/functional/modules/test_zos_lineinfile_func.py Co-authored-by: Fernando Flores * Update changelogs/fragments/1842-Add_test_case_to_validate_advance_regular_expression.yml Co-authored-by: Fernando Flores * Update test_zos_lineinfile_func.py * Validate use of members * Add pds pdse and member fails * Fix sanity issues * Fix error message and space_Type in lower case * Change names * Add docstring * Add documentation * fix upper case * Add comments * Add ignore * Add documentation * Try to fix the lint * Final test case * Fix two runs * Delete uneccesary validation * Fix test case and add to github templates * Fix error return message * Add test case * Return functions * Add member * Add test case * Add test case * Add missing import * Add missing import * Add documentation * Fix playbook * Fix error test case * Fix version of inventory * Remove repeitive test case * Update plugins/module_utils/zfsadm.py Co-authored-by: Alex Moreno * Add validation to avoid no auto increment * Fix a comment * Fix documentation * Apply suggestions from code review Co-authored-by: Alex Moreno * Add notes * Scape points * Scape points * Scape points * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Update plugins/modules/zos_zfs_resize.py Co-authored-by: Fernando Flores * Add 2025 and adjust to comments * Add notes * Fix errors * Fix documentation problem * Fix comment and test case * Fix comment * Fix comment * Fix comment * Fix error when creating one * Fix error when creating one * Fix error messaging * Fix documentation * Fix output * Add extra validations * Fix resize * Ensure proper size creation * Fix eror assertion * Add documenation and full verification * Fix PDS and PDSE * Fix documentation * Fix validation * Fix documentation and stderr for SEQ dataset * Update documentation * Fix names * Fix lint * Update word increment to increase Signed-off-by: ddimatos * Fix all type of names for uss trace destination * Check trace uss on other size --------- Signed-off-by: ddimatos Co-authored-by: Fernando Flores Co-authored-by: Rich Parker Co-authored-by: Demetri Co-authored-by: Alex Moreno --- .github/ISSUE_TEMPLATE/bug_issue.yml | 1 + .../ISSUE_TEMPLATE/collaboration_issue.yml | 1 + .github/ISSUE_TEMPLATE/doc_issue.yml | 1 + .github/ISSUE_TEMPLATE/enabler_issue.yml | 1 + .../enhancement_feature.issue.yml | 1 + plugins/module_utils/zfsadm.py | 84 ++ plugins/modules/zos_zfs_resize.py | 730 +++++++++++++ .../modules/test_zos_zfs_resize_func.py | 974 ++++++++++++++++++ tests/sanity/ignore-2.14.txt | 1 + tests/sanity/ignore-2.15.txt | 1 + tests/sanity/ignore-2.16.txt | 1 + tests/sanity/ignore-2.17.txt | 1 + 12 files changed, 1797 insertions(+) create mode 100644 plugins/module_utils/zfsadm.py create mode 100644 plugins/modules/zos_zfs_resize.py create mode 100644 tests/functional/modules/test_zos_zfs_resize_func.py diff --git a/.github/ISSUE_TEMPLATE/bug_issue.yml b/.github/ISSUE_TEMPLATE/bug_issue.yml index 0c0b578eb..f19bce9b1 100644 --- a/.github/ISSUE_TEMPLATE/bug_issue.yml +++ b/.github/ISSUE_TEMPLATE/bug_issue.yml @@ -139,6 +139,7 @@ body: - zos_tso_command - zos_unarchive - zos_volume_init + - zos_zfs_resize validations: required: false - type: textarea diff --git a/.github/ISSUE_TEMPLATE/collaboration_issue.yml b/.github/ISSUE_TEMPLATE/collaboration_issue.yml index b9505fe93..57c4200bb 100644 --- a/.github/ISSUE_TEMPLATE/collaboration_issue.yml +++ b/.github/ISSUE_TEMPLATE/collaboration_issue.yml @@ -146,5 +146,6 @@ body: - zos_tso_command - zos_unarchive - zos_volume_init + - zos_zfs_resize validations: required: false diff --git a/.github/ISSUE_TEMPLATE/doc_issue.yml b/.github/ISSUE_TEMPLATE/doc_issue.yml index 395567642..9370bdc99 100644 --- a/.github/ISSUE_TEMPLATE/doc_issue.yml +++ b/.github/ISSUE_TEMPLATE/doc_issue.yml @@ -83,5 +83,6 @@ body: - zos_tso_command - zos_unarchive - zos_volume_init + - zos_zfs_resize validations: required: false diff --git a/.github/ISSUE_TEMPLATE/enabler_issue.yml b/.github/ISSUE_TEMPLATE/enabler_issue.yml index c9584acfd..c8ed3986a 100644 --- a/.github/ISSUE_TEMPLATE/enabler_issue.yml +++ b/.github/ISSUE_TEMPLATE/enabler_issue.yml @@ -53,6 +53,7 @@ body: - zos_tso_command - zos_unarchive - zos_volume_init + - zos_zfs_resize validations: required: false diff --git a/.github/ISSUE_TEMPLATE/enhancement_feature.issue.yml b/.github/ISSUE_TEMPLATE/enhancement_feature.issue.yml index 98adbf65b..610cfb6ba 100644 --- a/.github/ISSUE_TEMPLATE/enhancement_feature.issue.yml +++ b/.github/ISSUE_TEMPLATE/enhancement_feature.issue.yml @@ -51,6 +51,7 @@ body: - zos_tso_command - zos_unarchive - zos_volume_init + - zos_zfs_resize validations: required: true diff --git a/plugins/module_utils/zfsadm.py b/plugins/module_utils/zfsadm.py new file mode 100644 index 000000000..3bcc077c5 --- /dev/null +++ b/plugins/module_utils/zfsadm.py @@ -0,0 +1,84 @@ +# Copyright (c) IBM Corporation 2025 +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import absolute_import, division, print_function +__metaclass__ = type + + +class zfsadm: + """Provides an interface to execute zfsadm commands. + """ + def __init__(self, aggregate_name, module): + """Initialize a class with its zfs dataset and allow modules to be executed. + Parameters + ---------- + aggregate_name : str + Name of the zfs dataset. + module : object + Ansible object to execute commands. + """ + self.aggregate_name = aggregate_name.upper() + self.module = module + + def execute_resizing(self, operation, size, noai, verbose): + """Execute a grow or shrink operation on a zfs dataset. + + Parameters + ---------- + operation : str + Whether the operation to execute is grow or shrink + size : int + Size to be assigned to the zfs + noai : str + Value for the no auto increase (noai) option when shrinking a dataset + verbose : str + Value for zfsadm's trace option specifying the output file for the command + + Returns + ------- + rc : int + The RC of the command executed. + stdout : str + The stdout of the executed command. + stderr : str + The stderr of the executed command. + cmd_str : str + The full command that was executed. + """ + if operation != "grow": + if operation != "shrink": + self.module.fail_json(msg=f"There is no operator {operation}") + + cmd = f"-size {size}{noai}{verbose}" + cmd_str = f"zfsadm {operation} -aggregate {self.aggregate_name} {cmd}" + + rc, stdout, stderr = self.module.run_command(cmd_str) + + return rc, stdout, stderr, cmd_str + + @staticmethod + def get_aggregate_size(aggregate_name, module): + """Execute a command to get the size of a zfs dataset. + + Returns + ------- + rc : int + The rc of the executed command. + stdout : str + The stdout of the executed command + stderr : str + The stderr of the executed command. + """ + cmd = f"zfsadm aggrinfo {aggregate_name}" + + rc, stdout, stderr = module.run_command(cmd) + + return rc, stdout, stderr diff --git a/plugins/modules/zos_zfs_resize.py b/plugins/modules/zos_zfs_resize.py new file mode 100644 index 000000000..162b10a3f --- /dev/null +++ b/plugins/modules/zos_zfs_resize.py @@ -0,0 +1,730 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- + +# Copyright (c) IBM Corporation 2025 +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import absolute_import, division, print_function +__metaclass__ = type + +DOCUMENTATION = r""" +module: zos_zfs_resize +version_added: '1.13.0' +short_description: Resize a zfs data set. +description: + - The module L(zos_zfs_resize.,/zos_zfs_resize.html) can resize a zFS aggregate data set. + - The I(target) data set must be a unique and a Fully Qualified Name (FQN) of a z/OS zFS aggregate data set. + - The data set must be attached as read-write. + - I(size) must be provided. +author: + - "Rich Parker (@richp405)" + - "Marcel Gutierrez (@AndreMarcel99)" +options: + target: + description: + - The Fully Qualified Name of the zFS data set that will be resized. + required: true + type: str + aliases: [ src ] + size: + description: + - The desired size of the data set after the resizing is performed. + required: true + type: int + space_type: + description: + - The unit of measurement to use when defining the size. + - Valid units are C(k) (kilobytes), C(m) (megabytes), C(g) (gigabytes), C(cyl) (cylinders), and C(trk) (tracks). + required: false + type: str + choices: + - k + - m + - g + - cyl + - trk + default: k + no_auto_increase: + description: + - Option controls whether the data set size will be automatically increased when performing a shrink operation. + - When set to C(true), during the shrinking of the zFS aggregate, if more space be needed the total size will + not be increased and the module will fail. + required: false + type: bool + default: false + verbose: + description: + - Return diagnostic messages that describe the module's execution. + - Verbose includes standard out (stdout) of the module's execution which can be excessive, to avoid writing + this to stdout, optionally you can set the C(trace_destination) instead. + required: false + type: bool + default: false + trace_destination: + description: + - Specify a unique USS file name or data set name for C(trace_destination). + - If the destination C(trace_destination) is a USS file or path, the C(trace_destination) must + be an absolute path name. + - Support MVS data set type SEQ, PDS, PDS/E(MEMBER) + - If the destination is an MVS data set name, the C(trace_destination) provided must meet data set naming + conventions of one or more qualifiers, each from one to eight characters long, that are delimited by periods. + required: false + type: str + +notes: + - If needed, allocate the zFS trace output data set as a PDSE with RECFM=VB, LRECL=133 with a primary allocation of at least + 50 cylinders and a secondary allocation of 30 cylinders. + - L(zfsadm documentation,https://www.ibm.com/docs/en/zos/latest?topic=commands-zfsadm). +""" + +EXAMPLES = r""" +- name: Resize an aggregate data set to 2500 kilobytes. + zos_zfs_resize: + target: TEST.ZFS.DATA + size: 2500 + +- name: Resize an aggregate data set to 20 tracks. + zos_zfs_resize: + target: TEST.ZFS.DATA + space_type: trk + size: 20 + +- name: Resize an aggregate data set to 4 megabytes. + zos_zfs_resize: + target: TEST.ZFS.DATA + space_type: m + size: 4 + +- name: Resize an aggregate data set to 1000 kilobytes and set no auto increase if it's shrinking. + zos_zfs_resize: + target: TEST.ZFS.DATA + size: 1000 + no_auto_increase: true + +- name: Resize an aggregate data set and get verbose output. + zos_zfs_resize: + target: TEST.ZFS.DATA + size: 2500 + verbose: true + +- name: Resize an aggregate data set and get the full trace on a file. + zos_zfs_resize: + target: TEST.ZFS.DATA + size: 2500 + trace_destination: /tmp/trace.txt + +- name: Resize an aggregate data set and capture the trace into a PDS member. + zos_zfs_resize: + target: TEST.ZFS.DATA + size: 2500 + trace_destination: "TEMP.HELPER.STORAGE(RESIZE)" + +- name: Resize an aggregate data set and capture the trace into a file with verbose output. + zos_zfs_resize: + target: TEST.ZFS.DATA + size: 2500 + verbose: true + trace_destination: /tmp/trace.txt +""" + +RETURN = r""" +cmd: + description: The zfsadm command executed on the remote node. + returned: always + type: str + sample: zfsadm grow -aggregate SOMEUSER.VVV.ZFS -size 4096 +target: + description: The Fully Qualified Name of the resized zFS data set. + returned: always + type: str + sample: SOMEUSER.VVV.ZFS +mount_target: + description: The original share/mount of the data set. + returned: always + type: str + sample: /tmp/zfs_agg +size: + description: The desired size from option C(size) according to C(space_type). + The resulting size can vary slightly, the actual space utilization is returned in C(new_size). + returned: always + type: int + sample: 4024 +rc: + description: The return code of the zfsadm command. + returned: always + type: int + sample: 0 +old_size: + description: The original data set size according to C(space_type) before resizing was performed. + returned: always + type: float + sample: 3096 +old_free_space: + description: The original data sets free space according to C(space_type) before resizing was performed. + returned: always + type: float + sample: 2.1 +new_size: + description: The data set size according to C(space_type) after resizing was performed. + returned: success + type: float + sample: 4032 +new_free_space: + description: The data sets free space according to C(space_type) after resizing was performed. + returned: success + type: float + sample: 1.5 +space_type: + description: The measurement unit of space used to report all size values. + returned: always + type: str + sample: k +stdout: + description: The modules standard out (stdout) that is returned. + returned: always + type: str + sample: IOEZ00173I Aggregate TEST.ZFS.DATA.USER successfully grown. +stderr: + description: The modules standard error (stderr) that is returned. it may have no return value. + returned: always + type: str + sample: IOEZ00181E Could not open trace output dataset. +stdout_lines: + description: List of strings containing individual lines from standard out (stdout). + returned: always + type: list + sample: ["IOEZ00173I Aggregate TEST.ZFS.DATA.USER successfully grown."] +stderr_lines: + description: List of strings containing individual lines from standard error (stderr). + returned: always + type: list + sample: ["IOEZ00181E Could not open trace output dataset."] +verbose_output: + description: If C(verbose=true), the operation's full traceback will show for this property. + returned: always + type: str + sample: 6FB2F8 print_trace_table printing contents of table Main Trace Table... +""" + +import os +import tempfile +from pathlib import Path +from ansible.module_utils.basic import AnsibleModule +from ansible_collections.ibm.ibm_zos_core.plugins.module_utils import ( + better_arg_parser, + data_set, +) + +from ansible_collections.ibm.ibm_zos_core.plugins.module_utils.zfsadm import zfsadm + + +def calculate_size_on_k(size, space_type): + """Function to convert size depending on the space_type. + + Parameters + ---------- + size : int + Size to grow or shrink the zfs + space_type : str + Type of space to be use + + Returns + ------- + size : int + Size on kilobytes + """ + if space_type == "m": + size *= 1024 + if space_type == "g": + size *= 1048576 + if space_type == "cyl": + size *= 830 + if space_type == "trk": + size *= 55 + return size + + +def get_full_output(file, module): + """Function to get the verbose output and delete the tmp file + + Parameters + ---------- + file : str + Name of the tmp file where the full verbose output is store + module : object + Ansible object to execute commands. + + Returns + ------- + output : str + Verbose output + """ + output = "" + + if "/" in file: + cmd = f"cat {file}" + else: + cmd = f"dcat '{file}'" + + rc, output, stderr = module.run_command(cmd) + + if rc != 0: + output = "Unable to obtain full output for verbose mode." + + return output + + +def get_size_and_free(line): + """Function to parsing the response of get aggregation size. + + Parameters + ---------- + line : str + stout of the get aggregation size function + + Returns + ------- + numbers[1] : int + Size on kilobytes of the zfs + + numbers[0] : int + Total free on kilobytes of the zfs + """ + numbers = [int(word) for word in line.split() if word.isdigit()] + return numbers[1], numbers[0] + + +def find_mount_target(module, target, results): + """Execute df command to access the information of mount points. + + Parameters + ---------- + module : object + Ansible object to execute commands. + target : str + ZFS to check. + + Returns + ------- + mount_point : str + The folder where the zfs is mount + """ + rc, stdout, stderr = module.run_command("df") + found = False + if rc == 0: + stdout_lines = stdout.split("\n") + for line in stdout_lines: + if len(line) > 2: + columns = line.split() + if target in columns[1]: + mount_point = columns[0] + found = True + break + if found is False: + module.fail_json(msg=f"No mount points were found in the following output: {stdout}", **results) + return mount_point + + +def convert_size(size, space_type): + """Function to convert size from kb to the space_type. + + Parameters + ---------- + size : int + Size to grow or shrink the zfs + space_type : str + Type of space to be use + + Returns + ------- + size : int + Size on kilobytes + """ + if space_type == "m": + size /= 1024 + if space_type == "g": + size /= 1048576 + if space_type == "cyl": + size /= 830 + if space_type == "trk": + size /= 55 + return size + + +def proper_size_str(size, space_type): + """Function to convert size to a proper response for output. + + Parameters + ---------- + size : int + Size to grow or shrink the zfs + space_type : str + Type of space to be use + + Returns + ------- + size : int or float + Size on specific space_type notation 2 if is 2.0 or 1.5 + """ + if space_type == "k" or space_type == "trk" or space_type == "cyl": + return int(size) + else: + split_size = str(size).split(".") + if split_size[1].startswith("0"): + return int(size) + else: + return float("{:.1f}".format(size)) + + +def create_trace_dataset(name, member=False): + """Function to create data sets for traceback if is not created. + + Parameters + ---------- + name : str + Full size of the dataset + member : bool + If the dataset include a member to create + + Returns + ------- + rc : bool + Indicates if datasets were made. + """ + if member: + dataset_name = data_set.extract_dsname(name) + data_set.DataSet.ensure_present(name=dataset_name, replace=False, type="PDSE", record_length=400) + rc = data_set.DataSet.ensure_member_present(name) + else: + rc = data_set.DataSet.ensure_present(name=name, replace=False, type="PDS", record_length=400) + + return rc + + +def run_module(): + module = AnsibleModule( + argument_spec=dict( + target=dict(type="str", required=True, aliases=['src']), + size=dict(type="int", required=True), + space_type=dict( + type="str", + required=False, + choices=["k", "m", "g", "cyl", "trk"], + default="k", + ), + no_auto_increase=dict(type="bool", required=False, default=False), + verbose=dict(type="bool", required=False, default=False), + trace_destination=dict(type="str", required=False), + ), + supports_check_mode=False + ) + args_def = dict( + target=dict(type="data_set", required=True, aliases=['src']), + size=dict(type="int", required=True), + space_type=dict( + type="str", + required=False, + choices=["k", "m", "g", "cyl", "trk"], + default="k", + ), + no_auto_increase=dict(type="bool", required=False, default=False), + verbose=dict(type="bool", required=False, default=False), + trace_destination=dict(type="data_set_or_path", required=False), + ) + + try: + parser = better_arg_parser.BetterArgParser(args_def) + parsed_args = parser.parse_args(module.params) + module.params = parsed_args + except ValueError as err: + module.fail_json( + msg='Parameter verification failed.', + stderr=str(err) + ) + + result = dict() + target = module.params.get("target") + size = module.params.get("size") + space_type = module.params.get("space_type") + noai = module.params.get("no_auto_increase") + verbose = module.params.get("verbose") + trace_destination = module.params.get("trace_destination") + + changed = False + # Variables to return the value on the space_type by the user + size_on_type = "" + free_on_type = "" + + result.update( + dict( + target=target, + size=size, + space_type=space_type, + cmd="", + changed=changed, + rc=1, + stdout="", + stderr="", + verbose_output="", + ) + ) + + if module.check_mode: + module.exit_json(**result) + + # Validate if the target zFS exist + if not (data_set.DataSet.data_set_exists(target)): + module.fail_json(msg=f"zFS Target {target} does not exist", **result) + + # Validation to found target on the system and also get the mount_point + mount_target = find_mount_target(module=module, target=target, results=result) + + if size <= 0: + module.fail_json(msg=f"Can not resize zFS aggregate Target {target} to 0", **result) + + # Initialize the class with the target + zfsadm_obj = zfsadm(aggregate_name=target, module=module) + + rc, stdout, stderr = zfsadm.get_aggregate_size(zfsadm_obj.aggregate_name, module) + + if rc == 0: + old_size, old_free = get_size_and_free(line=stdout) + + if space_type != "k": + space = calculate_size_on_k(size=size, space_type=space_type) + else: + space = size + + # Validations to know witch function will be execute + operation = "" + minimum_size_t_shrink = old_size - old_free + + if space_type != "k": + size_on_type = convert_size(size=old_size, space_type=space_type) + free_on_type = convert_size(size=old_free, space_type=space_type) + + str_old_size = old_size if space_type == "k" else size_on_type + str_old_size = proper_size_str(str_old_size, space_type) + str_old_free = old_free if space_type == "k" else free_on_type + str_old_free = proper_size_str(str_old_free, space_type) + + result.update( + dict( + mount_target=mount_target, + old_size=str_old_size, + old_free_space=str_old_free, + ) + ) + + if space == old_size: + result.update( + dict( + cmd="", + rc=0, + stdout=f"Size provided is the current size of the zFS {target}", + stderr="", + changed=False, + size=size, + new_size=str_old_size, + new_free_space=str_old_free, + ) + ) + module.exit_json(**result) + + elif space < minimum_size_t_shrink: + module.fail_json(msg="There is not enough available space in the zFS aggregate to perform a shrink operation.", **result) + + elif space > old_size: + operation = "grow" + + elif space >= minimum_size_t_shrink and space < old_size: + operation = "shrink" + + noai = " -noai " if noai else "" + noai = "" if operation == "grow" else noai + + # Variables for the verbose output or trace destination + trace = "" + tmp_file = "" + trace_uss = True + trace_destination_created = True + trace_type = "" + + if trace_destination is not None: + if data_set.is_data_set(data_set.extract_dsname(trace_destination)): + if data_set.is_member(trace_destination): + if not data_set.DataSet.data_set_exists(data_set.extract_dsname(trace_destination)): + trace_destination_created = create_trace_dataset(name=trace_destination, member=True) + else: + if not (data_set.DataSet.data_set_exists(trace_destination)): + trace_destination_created = create_trace_dataset(name=trace_destination, member=False) + else: + trace_type = data_set.DataSet.data_set_type(trace_destination) + trace_uss = False + else: + trace_destination = better_arg_parser.BetterArgHandler.fix_local_path(trace_destination) + trace_uss = True + tmp_file = trace_destination + + if not trace_destination_created: + stderr_trace = f"\nUnable to create trace_destination {trace_destination}." + else: + stderr_trace = "" + + if verbose and trace_destination is None: + home_folder = Path.home() + tmp_fld = module._remote_tmp.replace("~", str(home_folder)) + tmp_fld = tmp_fld.replace("//", "/") + temp = tempfile.NamedTemporaryFile(dir=tmp_fld, delete=False) + tmp_file = temp.name + trace_uss = True + + if verbose or trace_destination is not None: + trace = f" -trace '{tmp_file}'" if trace_uss else f" -trace \"//'{trace_destination}'\" " + + # Execute the function + rc, stdout, stderr, cmd = zfsadm_obj.execute_resizing(operation=operation, size=space, noai=noai, verbose=trace) + + # Get the output, calculate size and verbose if required + if rc == 0: + changed = True + + if "IOEZ00181E Could not open trace output dataset" in stderr and trace_type == "PS": + stderr = "" + + rc_size, stdout_size, stderr_size = zfsadm.get_aggregate_size(zfsadm_obj.aggregate_name, module) + if rc_size == 0: + new_size, new_free = get_size_and_free(line=stdout_size) + + if space_type != "k": + size_on_type = convert_size(size=new_size, space_type=space_type) + free_on_type = convert_size(size=new_free, space_type=space_type) + + if verbose: + if trace_destination is None: + output = get_full_output(file=tmp_file, module=module) + result.update( + verbose_output=output, + ) + os.remove(tmp_file) + else: + output = get_full_output(file=tmp_file, module=module) + result.update( + verbose_output=output, + ) + + else: + if verbose and trace_destination is None: + os.remove(tmp_file) + + msg = "No enough space on device to grow." if operation == 'grow' else "No space to properly shrink." + + raise ResizingOperationError( + msg=f"Resize: resize command returned non-zero code. {msg}", + target=target, + mount_target=mount_target, + cmd=cmd, + rc=rc, + size=size, + stdout=stdout, + stderr=stderr + stderr_trace, + changed=False, + old_size=str_old_size, + old_free=str_old_free, + verbose_output="", + ) + + str_new_size = new_size if space_type == "k" else size_on_type + str_new_size = proper_size_str(str_new_size, space_type) + str_new_free = new_free if space_type == "k" else free_on_type + str_new_free = proper_size_str(str_new_free, space_type) + + result.update( + dict( + cmd=cmd, + rc=rc, + stdout=stdout, + stderr=stderr + stderr_trace, + changed=changed, + new_size=str_new_size, + new_free_space=str_new_free, + ) + ) + + module.exit_json(**result) + + +class ResizingOperationError(Exception): + def __init__( + self, + msg, + target="", + mount_target="", + size="", + cmd="", + rc="", + stdout="", + stderr="", + changed=False, + old_size="", + old_free="", + verbose_output="", + ): + """Error in a copy operation. + + Parameters + ---------- + msg : str + Human readable string describing the exception. + target : str + The Fully Qualified Name of the resized zfs data set + mount_target : str + The original share/mount + size : str + The approximate size of the target + rc : int + Result code. + stdout : str + Standart output. + stderr : str + Standart error. + cmd : str + The zfsadm command try to execute on the remote node. + changed : bool + If the operation was executed. + old_size : float or int + The reported size, of the data set before the resizing is performed. + old_free : float or int + The reported size, of the free space in the data set before the resizing is performed. + """ + self.json_args = dict( + msg=msg, + target=target, + mount_target=mount_target, + cmd=cmd, + rc=rc, + size=size, + stdout=stdout, + stderr=stderr, + changed=changed, + old_size=old_size, + old_free_space=old_free, + verbose_output=verbose_output, + ) + super().__init__(msg) + + +def main(): + run_module() + + +if __name__ == '__main__': + main() diff --git a/tests/functional/modules/test_zos_zfs_resize_func.py b/tests/functional/modules/test_zos_zfs_resize_func.py new file mode 100644 index 000000000..d6bd5898e --- /dev/null +++ b/tests/functional/modules/test_zos_zfs_resize_func.py @@ -0,0 +1,974 @@ +#!/usr/bin/python +# -*- coding: utf-8 -*- + +# Copyright (c) IBM Corporation 2025 +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# http://www.apache.org/licenses/LICENSE-2.0 +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +from __future__ import absolute_import, division, print_function + +import pytest +import os +import yaml +from shellescape import quote + +from ibm_zos_core.tests.helpers.dataset import get_tmp_ds_name + +from ibm_zos_core.tests.helpers.utils import get_random_file_name + +__metaclass__ = type + +NO_AUTO_INCREMENT= """- hosts : zvm + collections : + - ibm.ibm_zos_core + gather_facts: False + vars: + ZOAU: "{0}" + PYZ: "{1}" + environment: + _BPXK_AUTOCVT: "ON" + ZOAU_HOME: "{0}" + PYTHONPATH: "{0}/lib/{2}" + LIBPATH: "{0}/lib:{1}/lib:/lib:/usr/lib:." + PATH: "{0}/bin:/bin:/usr/lpp/rsusr/ported/bin:/var/bin:/usr/lpp/rsusr/ported/bin:/usr/lpp/java/java180/J8.0_64/bin:{1}/bin:" + _CEE_RUNOPTS: "FILETAG(AUTOCVT,AUTOTAG) POSIX(ON)" + _TAG_REDIR_ERR: "txt" + _TAG_REDIR_IN: "txt" + _TAG_REDIR_OUT: "txt" + LANG: "C" + tasks: + - name: Create ZFS. + block: + - name: Create data set to ZFS + zos_data_set: + name: {3} + type: zfs + space_primary: 1 + space_type: m + replace: true + + - name: Create mount dir on z/OS USS. + file: + path: {4} + state: directory + + - name: Mount ZFS data set. + command: /usr/sbin/mount -t zfs -f {3} {4} + + - name: Create folder. + shell: mkdir {4}/folder + + - name: Create a folder. + shell: touch {4}/folder/test.txt + - name: Write bytes on file. + shell: head -c 150000 /dev/urandom > {4}/folder/test.txt + + - name: Create a folder. + shell: touch {4}/folder/test1.txt + - name: Write bytes on file. + shell: head -c 100000 /dev/urandom > {4}/folder/test1.txt + + - name: Create a folder. + shell: touch {4}/folder/test3.txt + - name: Write bytes on file. + shell: head -c 1000000 /dev/urandom > {4}/folder/test3.txt + + - name: Create a folder. + shell: touch {4}/folder/test4.txt + - name: Write bytes on file. + shell: head -c 100000 /dev/urandom > {4}/folder/test4.txt + + - name: Create a folder. + shell: touch {4}/folder/test5.txt + - name: Write bytes on file. + shell: head -c 100000 /dev/urandom > {4}/folder/test5.txt + + - name: Create a folder. + shell: touch {4}/folder/test7.txt + - name: Write bytes on file. + shell: head -c 100000 /dev/urandom > {4}/folder/test7.txt + + - name: Remove a folder. + shell: rm {4}/folder/test3.txt + + - name: Shrink ZFS aggregate in k and auto_increment. + zos_zfs_resize: + target: {3} + size: 900 + no_auto_increase: {5} + poll: 0 + register: shrink_output + + - name: Create a folder. + shell: touch {4}/folder/test6.txt + - name: Write bytes on file. + shell: head -c 100000 /dev/urandom > {4}/folder/test6.txt + poll: 0 + + always: + - name: Unmount ZFS data set. + command: /usr/sbin/unmount {4} + + - name: Delete ZFS data set. + zos_data_set: + name: {3} + state: absent + + - name: Unmount ZFS data set. + command: rm -r {4} +""" + +INVENTORY = """all: + hosts: + zvm: + ansible_host: {0} + ansible_ssh_private_key_file: {1} + ansible_user: {2} + ansible_python_interpreter: {3}/bin/python{4}""" + +def make_temp_folder(hosts): + """Create a temporary file on a z/OS system and return its path.""" + tempfile_name = "" + results = hosts.all.tempfile(state="directory") + for result in results.contacted.values(): + tempfile_name = result.get("path", "") + return tempfile_name + +def set_environment(ansible_zos_module, ds_name, space=1, space_type='m'): + """Create a ZFS data set, mount it to a temp folder and fill it with test data. + + Parameters + ---------- + ansible_zos_module : object + Ansible object to execute commands. + ds_name : str + ZFS name. + space : int + space of ZFS data set. + space_type : str + space type used to create the ZFS. + + Returns + ------- + temp_dir_name : str + The folder where the zfs is mounted. + """ + hosts = ansible_zos_module + + hosts.all.zos_data_set(name=ds_name, type="zfs", space_primary=space, space_type=space_type) + temp_dir_name = make_temp_folder(hosts=hosts) + hosts.all.command( + cmd="usr/sbin/mount -t zfs -f {0} {1}".format( + ds_name, + temp_dir_name + ) + ) + if space_type == "m": + bits_wr = 1000000 + else: + bits_wr = 10000 + + hosts.all.command( + cmd="head -c {0} /dev/urandom > {1}/test.txt".format( + bits_wr, + temp_dir_name + ) + ) + + return temp_dir_name + +def clean_up_environment(hosts, ds_name, temp_dir_name): + """Unmount a ZFS dataset, delete it and delete the folder on which it was mounted. + + Parameters + ---------- + hosts : object + Ansible object to execute commands. + ds_name : str + ZFS name. + temp_dir_name : str + Folder where the ZFS is mounted. + """ + hosts.all.command(cmd=f"usr/sbin/unmount {temp_dir_name}") + hosts.all.zos_data_set(name=ds_name, state="absent") + hosts.all.file(path=temp_dir_name, state="absent") + +######################### +# Positive test cases +######################### + +def test_grow_operation(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 2500 + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('new_size') >= size + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_shrink_operation(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 1200 + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('new_size') <= size + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_grow_n_shrink_operations_space_type_m(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 3 + shrink_size = 2 + space_type = "m" + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == space_type + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('space_type') == space_type + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('new_size') >= grow_size + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == space_type + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('space_type') == space_type + assert result.get('new_size') <= shrink_size + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_grow_n_shrink_operations_space_type_trk(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 400 + shrink_size = 200 + space_type = "trk" + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == space_type + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= grow_size + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('space_type') == space_type + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == space_type + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('new_size') <= shrink_size + assert result.get('space_type') == space_type + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_grow_n_shrink_operations_space_type_cyl(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 3 + shrink_size = 2 + space_type = "cyl" + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == space_type + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('new_size') >= grow_size + assert result.get('space_type') == space_type + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == space_type + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('new_size') <= shrink_size + assert result.get('space_type') == space_type + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_grow_n_shrink_operations_verbose(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 1800 + shrink_size = 1200 + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + verbose=True) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == "k" + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= grow_size + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('space_type') == "k" + assert result.get("verbose_output") is not None + assert "Printing contents of table at address" in result.get("stdout") + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + verbose=True) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == "k" + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= grow_size + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('space_type') == "k" + assert result.get("verbose_output") is not None + assert "print of in-memory trace table has completed" in result.get('stdout') + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_grow_n_shrink_operations_trace_uss(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 1800 + shrink_size = 1200 + trace_destination_file = "/" + get_random_file_name(dir="tmp") + trace_destination_file_s = "/" + get_random_file_name(dir="tmp") + try: + hosts.all.shell(cmd="touch {0}".format(trace_destination_file)) + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + trace_destination=trace_destination_file) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= grow_size + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "Printing contents of table at address" in result.get("stdout") + cmd = "cat {0}".format(trace_destination_file) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + hosts.all.shell(cmd="touch {0}".format(trace_destination_file_s)) + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + trace_destination=trace_destination_file_s) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= grow_size + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "print of in-memory trace table has completed" in result.get('stdout') + cmd = "cat {0}".format(trace_destination_file_s) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + finally: + hosts.all.shell(cmd="rm {0}".format(trace_destination_file)) + hosts.all.shell(cmd="rm {0}".format(trace_destination_file_s)) + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_grow_n_shrink_operations_trace_uss_not_created(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 1800 + shrink_size = 1200 + trace_destination_file = "/" + get_random_file_name(dir="tmp") + trace_destination_file_s = "/" + get_random_file_name(dir="tmp") + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + trace_destination=trace_destination_file) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= grow_size + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "Printing contents of table at address" in result.get("stdout") + cmd = "cat {0}".format(trace_destination_file) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + trace_destination=trace_destination_file_s) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= grow_size + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "print of in-memory trace table has completed" in result.get('stdout') + cmd = "cat {0}".format(trace_destination_file_s) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + finally: + hosts.all.shell(cmd="rm {0}".format(trace_destination_file)) + hosts.all.shell(cmd="rm {0}".format(trace_destination_file_s)) + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +@pytest.mark.parametrize("trace_destination", ["seq", "pds", "pdse"]) +def test_grow_n_shrink_operations_trace_ds(ansible_zos_module, trace_destination): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 1800 + shrink_size = 1200 + + trace_destination_ds = get_tmp_ds_name() + trace_destination_ds_s= get_tmp_ds_name() + try: + if trace_destination == "seq": + hosts.all.zos_data_set(name=trace_destination_ds, type=trace_destination, record_length=400) + else: + hosts.all.zos_data_set(name=trace_destination_ds, type=trace_destination, record_length=400) + trace_destination_ds = trace_destination_ds + "(MEM)" + hosts.all.zos_data_set(name=trace_destination_ds, state="present", type="member") + + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + trace_destination=trace_destination_ds) + + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= grow_size + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "Printing contents of table at address" in result.get("stdout") + cmd = "cat \"//'{0}'\" ".format(trace_destination_ds) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + if trace_destination == "seq": + hosts.all.zos_data_set(name=trace_destination_ds_s, type=trace_destination, record_length=400) + else: + hosts.all.zos_data_set(name=trace_destination_ds_s, type=trace_destination, record_length=400) + trace_destination_ds_s = trace_destination_ds_s + "(MEM)" + hosts.all.zos_data_set(name=trace_destination_ds_s, state="present", type="member") + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + trace_destination=trace_destination_ds_s) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= grow_size + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "print of in-memory trace table has completed" in result.get('stdout') + cmd = "cat \"//'{0}'\" ".format(trace_destination_ds_s) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + if trace_destination == "pds" or trace_destination == "pdse": + trace_destination_ds = trace_destination_ds.split("(")[0] + trace_destination_ds_s = trace_destination_ds_s.split("(")[0] + hosts.all.zos_data_set(name=trace_destination_ds, state="absent") + hosts.all.zos_data_set(name=trace_destination_ds_s, state="absent") + +@pytest.mark.parametrize("trace_destination", ["pds", "member"]) +def test_grow_n_shrink_operations_trace_ds_not_created(ansible_zos_module, trace_destination): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + grow_size = 1800 + shrink_size = 1200 + + trace_destination_ds = get_tmp_ds_name() + trace_destination_ds = trace_destination_ds if trace_destination == "pds" else trace_destination_ds + "(MEM)" + trace_destination_ds_s= get_tmp_ds_name() + trace_destination_ds_s = trace_destination_ds_s if trace_destination == "pds" else trace_destination_ds_s + "(MEM)" + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=grow_size, + trace_destination=trace_destination_ds) + + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == grow_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "grown" in result.get('stdout') + assert result.get('new_size') >= grow_size + assert result.get('new_size') >= result.get('old_size') + assert result.get('new_free_space') >= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "Printing contents of table at address" in result.get("stdout") + cmd = "cat \"//'{0}'\" ".format(trace_destination_ds) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + results = hosts.all.zos_zfs_resize(target=ds_name, + size=shrink_size, + trace_destination=trace_destination_ds_s) + for result in results.contacted.values(): + assert result.get('cmd') is not None + assert result.get('size') == shrink_size + assert result.get('space_type') == "k" + assert result.get('verbose_output') is not None + assert result.get('changed') is True + assert result.get('stdout_lines') is not None + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert "shrunk" in result.get('stdout') + assert result.get('new_size') <= grow_size + assert result.get('new_size') <= result.get('old_size') + assert result.get('new_free_space') <= result.get('old_free_space') + assert result.get('space_type') == "k" + assert "print of in-memory trace table has completed" in result.get('stdout') + cmd = "cat \"//'{0}'\" ".format(trace_destination_ds_s) + output_of_trace_file = hosts.all.shell(cmd=cmd) + for out in output_of_trace_file.contacted.values(): + assert out.get("stdout") is not None + assert result.get('stderr') == "" + assert result.get('stderr_lines') == [] + + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + if trace_destination == "pds" or trace_destination == "pdse": + trace_destination_ds = trace_destination_ds.split("(")[0] + trace_destination_ds_s = trace_destination_ds_s.split("(")[0] + hosts.all.zos_data_set(name=trace_destination_ds, state="absent") + hosts.all.zos_data_set(name=trace_destination_ds_s, state="absent") + +######################### +# Negative test cases +######################### + +@pytest.mark.parametrize("space_type", ["K", "TRK", "CYL", "M", "G"]) +def test_space_type_not_accept(ansible_zos_module, space_type): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 2500 + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size, + space_type=space_type) + for result in results.contacted.values(): + assert result.get('failed') == True + assert result.get('msg') == "value of space_type must be one of: k, m, g, cyl, trk, got: {0}".format(space_type) + +def test_target_does_not_exist(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 2500 + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size,) + for result in results.contacted.values(): + assert result.get('target') == ds_name + assert result.get('size') == size + assert result.get('rc') == 1 + assert result.get('changed') is False + assert result.get('msg') == "zFS Target {0} does not exist".format(ds_name) + +def test_mount_point_does_not_exist(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 2500 + try: + hosts.all.zos_data_set(name=ds_name, type="zfs", space_primary=1, space_type="m") + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size,) + for result in results.contacted.values(): + assert result.get('target') == ds_name + assert result.get('size') == size + assert result.get('rc') == 1 + assert result.get('changed') is False + assert "No mount points were found in the following output:" in result.get('msg') + finally: + hosts.all.zos_data_set(name=ds_name, state="absent") + +def test_no_operation_executed(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 1440 + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size,) + for result in results.contacted.values(): + assert result.get("changed") == False + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 0 + assert result.get('size') == size + assert result.get('space_type') == "k" + assert result.get('stdout') == "Size provided is the current size of the zFS {0}".format(ds_name) + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_no_space_to_operate(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 100 + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size,) + for result in results.contacted.values(): + assert result.get("changed") == False + assert result.get('target') == ds_name + assert result.get('mount_target') == "/SYSTEM" + mount_folder + assert result.get('rc') == 1 + assert result.get('size') == size + assert result.get('space_type') == "k" + assert result.get('msg') == "There is not enough available space in the zFS aggregate to perform a shrink operation." + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +def test_fail_operation(ansible_zos_module): + hosts = ansible_zos_module + ds_name = get_tmp_ds_name() + mount_folder = "" + size = 200 + try: + mount_folder = set_environment(ansible_zos_module=hosts, ds_name=ds_name) + results = hosts.all.zos_zfs_resize(target=ds_name, + size=size,) + for result in results.contacted.values(): + assert result.get("failed") == True + assert result.get('changed') == False + assert result.get('rc') == 1 + finally: + clean_up_environment(hosts=hosts, ds_name=ds_name, temp_dir_name=mount_folder) + +############################# +# No auto increment playbook +############################# + +def test_no_auto_increase(get_config): + ds_name = get_tmp_ds_name() + mount_point = "/" + get_random_file_name(dir="tmp") + path = get_config + with open(path, 'r') as file: + enviroment = yaml.safe_load(file) + ssh_key = enviroment["ssh_key"] + hosts = enviroment["host"].upper() + user = enviroment["user"].upper() + python_path = enviroment["python_path"] + cut_python_path = python_path[:python_path.find('/bin')].strip() + zoau = enviroment["environment"]["ZOAU_ROOT"] + python_version = cut_python_path.split('/')[2] + + try: + playbook = "playbook.yml" + inventory = "inventory.yml" + os.system("echo {0} > {1}".format(quote(NO_AUTO_INCREMENT.format( + zoau, + cut_python_path, + python_version, + ds_name, + mount_point, + "True" + )), playbook)) + os.system("echo {0} > {1}".format(quote(INVENTORY.format( + hosts, + ssh_key, + user, + cut_python_path, + python_version + )), inventory)) + command = "ansible-playbook -i {0} {1}".format( + inventory, + playbook + ) + stdout = os.system(command) + assert stdout != 0 + finally: + os.remove("inventory.yml") + os.remove("playbook.yml") + +def test_no_auto_increase_accept(get_config): + ds_name = get_tmp_ds_name() + mount_point = "/" + get_random_file_name(dir="tmp") + path = get_config + with open(path, 'r') as file: + enviroment = yaml.safe_load(file) + ssh_key = enviroment["ssh_key"] + hosts = enviroment["host"].upper() + user = enviroment["user"].upper() + python_path = enviroment["python_path"] + cut_python_path = python_path[:python_path.find('/bin')].strip() + zoau = enviroment["environment"]["ZOAU_ROOT"] + python_version = cut_python_path.split('/')[2] + + try: + playbook = "playbook.yml" + inventory = "inventory.yml" + os.system("echo {0} > {1}".format(quote(NO_AUTO_INCREMENT.format( + zoau, + cut_python_path, + python_version, + ds_name, + mount_point, + "False" + )), playbook)) + os.system("echo {0} > {1}".format(quote(INVENTORY.format( + hosts, + ssh_key, + user, + cut_python_path, + python_version + )), inventory)) + command = "ansible-playbook -i {0} {1}".format( + inventory, + playbook + ) + stdout = os.system(command) + assert stdout == 0 + finally: + os.remove("inventory.yml") + os.remove("playbook.yml") \ No newline at end of file diff --git a/tests/sanity/ignore-2.14.txt b/tests/sanity/ignore-2.14.txt index c04ae2328..51e44821a 100644 --- a/tests/sanity/ignore-2.14.txt +++ b/tests/sanity/ignore-2.14.txt @@ -23,3 +23,4 @@ plugins/modules/zos_gather_facts.py validate-modules:missing-gplv3-license # Lic plugins/modules/zos_volume_init.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_archive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_unarchive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 +plugins/modules/zos_zfs_resize.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 diff --git a/tests/sanity/ignore-2.15.txt b/tests/sanity/ignore-2.15.txt index c04ae2328..51e44821a 100644 --- a/tests/sanity/ignore-2.15.txt +++ b/tests/sanity/ignore-2.15.txt @@ -23,3 +23,4 @@ plugins/modules/zos_gather_facts.py validate-modules:missing-gplv3-license # Lic plugins/modules/zos_volume_init.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_archive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_unarchive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 +plugins/modules/zos_zfs_resize.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 diff --git a/tests/sanity/ignore-2.16.txt b/tests/sanity/ignore-2.16.txt index c04ae2328..51e44821a 100644 --- a/tests/sanity/ignore-2.16.txt +++ b/tests/sanity/ignore-2.16.txt @@ -23,3 +23,4 @@ plugins/modules/zos_gather_facts.py validate-modules:missing-gplv3-license # Lic plugins/modules/zos_volume_init.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_archive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_unarchive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 +plugins/modules/zos_zfs_resize.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 diff --git a/tests/sanity/ignore-2.17.txt b/tests/sanity/ignore-2.17.txt index 7ae119205..59f1b0a9f 100644 --- a/tests/sanity/ignore-2.17.txt +++ b/tests/sanity/ignore-2.17.txt @@ -23,3 +23,4 @@ plugins/modules/zos_gather_facts.py validate-modules:missing-gplv3-license # Lic plugins/modules/zos_volume_init.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_archive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 plugins/modules/zos_unarchive.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0 +plugins/modules/zos_zfs_resize.py validate-modules:missing-gplv3-license # Licensed under Apache 2.0