win_get_url doesn't get correct filename for URL's containing parameters #575

worshach · 2024-01-10T01:41:38Z

SUMMARY

URLs containing parameters fail to download properly. Only the part before the parameter gets downloaded..

ISSUE TYPE

Bug Report

COMPONENT NAME

win_get_url

ANSIBLE VERSION

ansible [core 2.15.3]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/lib/python3.11/site-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/bin/ansible
  python version = 3.11.5 (main, Nov 15 2023, 18:13:17) [GCC 8.5.0 20210514 (Red Hat 8.5.0-20.0.1)] (/usr/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

COLLECTION VERSION

[root@d1a1f04ad817 runner]# ansible-galaxy collection list ansible.windows

# /usr/share/ansible/collections/ansible_collections
Collection      Version
--------------- -------
ansible.windows 2.2.0

CONFIGURATION

[root@d1a1f04ad817 runner]# ansible-config dump --only-changed
CONFIG_FILE() = None

OS / ENVIRONMENT

Target OS version:
Microsoft Windows Server 2022
Version 21H2 (OS Build 20348.2159)

STEPS TO REPRODUCE

---
- name: Create temporary directory
  ansible.windows.win_tempfile:
    state: directory
  register: tempDir

- name: Build URL (using set_fact for troubleshooting)
      ansible.builtin.set_fact:
        url: "https://{{ ppdm_name }}/agent-download?filename={{ package_name }}"
  
- name: Windows package download
  ansible.windows.win_get_url:
    url: "{{ url }}"
    dest: "{{ tempDir['path'] }}"
    validate_certs: false
    headers:
      Authorization: "Bearer {{ access_token }}"

EXPECTED RESULTS

File downloaded to temp directory should be:  "C:\\Users\\ansible\\AppData\\Local\\Temp\\ansible.uoatn0ln.3aw\\fsagent1915_win_x64.zip"

Microsoft Windows [Version 10.0.20348.2159]
(c) Microsoft Corporation. All rights reserved.

C:\Windows\system32>cd c:\users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw

c:\Users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw>dir
 Volume in drive C has no label.
 Volume Serial Number is 1ACD-6551

 Directory of c:\Users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw

01/09/2024  05:31 PM    <DIR>          .
01/09/2024  05:31 PM    <DIR>          ..
               0 File(s)              0 bytes
               2 Dir(s)  75,151,159,296 bytes free

c:\Users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw>curl --insecure -o fsagent1915_win_x64.zip -H "Authorization: Bearer xxx" https://bakppdm01.vlab.local/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 82.4M  100 82.4M    0     0  75.1M      0  0:00:01  0:00:01 --:--:-- 75.2M

c:\Users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw>dir
 Volume in drive C has no label.
 Volume Serial Number is 1ACD-6551

 Directory of c:\Users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw

01/09/2024  05:36 PM    <DIR>          .
01/09/2024  05:31 PM    <DIR>          ..
01/09/2024  05:36 PM                 0 curl
01/09/2024  05:36 PM        86,428,441 **fsagent1915_win_x64.zip**
               2 File(s)     86,428,441 bytes
               2 Dir(s)  75,064,545,280 bytes free

c:\Users\ansible\AppData\Local\Temp\ansible.uaotn01n.3aw>

ACTUAL RESULTS

File downloaded to temp directory is: "C:\\Users\\ansible\\AppData\\Local\\Temp\\ansible.uoatn0ln.3aw\\agent-download"

TASK [Create temporary directory] **********************************************
task path: /runner/project/roles/download_files/tasks/windows.yml:7
changed: [win01.vlab.local] => {
    "changed": true,
    "invocation": {
        "module_args": {
            "path": "C:\\\\Users\\\\ansible\\\\AppData\\\\Local\\\\Temp",
            "prefix": "ansible.",
            "state": "directory",
            "suffix": null
        }
    },
    "path": "C:\\\\Users\\\\ansible\\\\AppData\\\\Local\\\\Temp\\\\ansible.uoatn0ln.3aw",
    "state": "directory"
}
Read vars_file './vars/defaults.yml'

TASK [Build URL] ***************************************************************
task path: /runner/project/roles/download_files/tasks/windows.yml:13
ok: [win01.vlab.local] => {
    "ansible_facts": {
        "url": "https://bakppdm01.vlab.local/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip"
    },
    "changed": false
}
Read vars_file './vars/defaults.yml'
Using module file /usr/share/ansible/collections/ansible_collections/ansible/windows/plugins/modules/win_get_url.ps1
Pipelining is enabled.
<win01.vlab.local> ESTABLISH WINRM CONNECTION FOR USER: [email protected] on PORT 5985 TO win01.vlab.local
EXEC (via pipeline wrapper)

TASK [Windows package download] ************************************************
task path: /runner/project/roles/download_files/tasks/windows.yml:17
changed: [win01.vlab.local] => {
    "changed": true,
    "checksum_dest": "67f20ea9382826613b70cd03c50dd803efd19146",
    "checksum_src": "67f20ea9382826613b70cd03c50dd803efd19146",
    "dest": "C:\\\\Users\\\\ansible\\\\AppData\\\\Local\\\\Temp\\\\ansible.uoatn0ln.3aw\\\\**agent-download**",
    "elapsed": 1.4687614,
    "invocation": {
        "module_args": {
            "checksum": null,
            "checksum_algorithm": "sha1",
            "checksum_url": null,
            "client_cert": null,
            "client_cert_password": null,
            "dest": "C:\\\\Users\\\\ansible\\\\AppData\\\\Local\\\\Temp\\\\ansible.uoatn0ln.3aw",
            "follow_redirects": "safe",
            "force": true,
            "force_basic_auth": false,
            "headers": {
                "Authorization": "Bearer xxx"
            },
            "http_agent": "ansible-httpget",
            "maximum_redirection": 50,
            "proxy_password": null,
            "proxy_url": null,
            "proxy_use_default_credential": false,
            "proxy_username": null,
            "url": "https://bakppdm01.vlab.local/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip",
            "url_method": null,
            "url_password": null,
            "url_timeout": 30,
            "url_username": null,
            "use_default_credential": false,
            "use_proxy": true,
            "validate_certs": false
        }
    },
    "msg": "OK",
    "size": 86428441,
    "status_code": 200,
    "url": "https://bakppdm01.vlab.local/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip"
}

The text was updated successfully, but these errors were encountered:

jborean93 · 2024-01-15T00:31:24Z

File downloaded to temp directory is: "C:\Users\ansible\AppData\Local\Temp\ansible.uoatn0ln.3aw\agent-download"

That's the expected results right now. If you specify dest as a path to a directory the code is going to use the path part of the request URL as the filename to download it to which in this case is agent-download. Essentially it is this logic:

$url = "https://foo/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip"
Split-Path -Path ([Uri]$url).LocalPath -Leaf

There is an open feature request to have win_get_url utilise the filename in the response Content-Disposition header but that has not been implemented yet #456. Even in your case the filename portion won't be used unless the HTTP response has Content-Disposition: attachment; filename="fsagent1915_win_x64.zip" in the response.

To safely download a file with a specific name use dest to a file or path that does not exist rather than the directory.

Here is a local example with my own webserver just returning a simple txt file

- hosts: SERVER2022
  gather_facts: false

  tasks:
  - name: clear temp folder 1
    ansible.windows.win_file:
      path: C:\temp\test
      state: '{{ item }}'
    loop:
    - absent
    - directory

  - name: download file with dest as dir
    ansible.windows.win_get_url:
      url: https://192.168.56.1:36021/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip
      dest: C:\temp\test
      validate_certs: false

  - name: show dir contents 1
    ansible.windows.win_shell: gci C:\temp\test

  - name: clear temp folder 2
    ansible.windows.win_file:
      path: C:\temp\test
      state: '{{ item }}'
    loop:
    - absent
    - directory

  - name: download file with dest with filename
    ansible.windows.win_get_url:
      url: https://192.168.56.1:36021/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip
      dest: C:\temp\test\fsagent1915_win_x64.zip
      validate_certs: false

  - name: show dir contents 2
    ansible.windows.win_shell: gci C:\temp\test

The output from this is

(ansible-3.11) jborean:~/dev/ansible-tester$ ansible-playbook main.yml -vv
ansible-playbook [core 2.17.0.dev0] (yaml-dt 259a6df398) last updated 2024/01/11 15:05:37 (GMT +1000)
  config file = /home/jborean/dev/ansible-tester/ansible.cfg
  configured module search path = ['/home/jborean/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /home/jborean/dev/ansible/lib/ansible
  ansible collection location = /home/jborean/dev:/home/jborean/ansible/collections:/user/share/ansible/collections
  executable location = /home/jborean/.pyenv/versions/ansible-3.11/bin/ansible-playbook
  python version = 3.11.5 (main, Sep  6 2023, 10:54:31) [GCC 13.2.1 20230801] (/home/jborean/.pyenv/versions/3.11.5/envs/ansible-3.11/bin/python3.11)
  jinja version = 3.1.2
  libyaml = True

PLAYBOOK: main.yml ************************************************************************
1 plays in main.yml

PLAY [SERVER2022] *************************************************************************

TASK [clear temp folder 1] ***********************************************************************************************************************************************************
task path: /home/jborean/dev/ansible-tester/main.yml:5
changed: [SERVER2022] => (item=absent) =>
    ansible_loop_var: item
    changed: true
    item: absent
changed: [SERVER2022] => (item=directory) =>
    ansible_loop_var: item
    changed: true
    item: directory

TASK [download file with dest as dir] ************************************************************************************************************************************************
task path: /home/jborean/dev/ansible-tester/main.yml:13
changed: [SERVER2022] =>
    changed: true
    checksum_dest: 83044656e0e13f7b2cbc501775b7260aa8684486
    checksum_src: 83044656e0e13f7b2cbc501775b7260aa8684486
    dest: C:\temp\test\agent-download
    elapsed: 0.09434999999999999
    msg: OK
    size: 1009
    status_code: 200
    url: https://192.168.56.1:36021/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip

TASK [show dir contents 1] ***********************************************************************************************************************************************************
task path: /home/jborean/dev/ansible-tester/main.yml:19
changed: [SERVER2022] =>
    changed: true
    cmd: gci C:\temp\test
    delta: '0:00:00.296674'
    end: '2024-01-15 10:28:12.319356'
    rc: 0
    start: '2024-01-15 10:28:12.022682'
    stderr: ''
    stderr_lines: <omitted>
    stdout: |4-


            Directory: C:\temp\test


        Mode                 LastWriteTime         Length Name
        ----                 -------------         ------ ----
        -a----         1/15/2024  10:28 AM           1009 agent-download
    stdout_lines: <omitted>

TASK [clear temp folder 2] ***********************************************************************************************************************************************************
task path: /home/jborean/dev/ansible-tester/main.yml:22
changed: [SERVER2022] => (item=absent) =>
    ansible_loop_var: item
    changed: true
    item: absent
changed: [SERVER2022] => (item=directory) =>
    ansible_loop_var: item
    changed: true
    item: directory

TASK [download file with dest with filename] *****************************************************************************************************************************************
task path: /home/jborean/dev/ansible-tester/main.yml:30
changed: [SERVER2022] =>
    changed: true
    checksum_dest: 83044656e0e13f7b2cbc501775b7260aa8684486
    checksum_src: 83044656e0e13f7b2cbc501775b7260aa8684486
    dest: C:\temp\test\fsagent1915_win_x64.zip
    elapsed: 0.0630925
    msg: OK
    size: 1009
    status_code: 200
    url: https://192.168.56.1:36021/agent-download?filename=19.15-69/Windows/FS/fsagent1915_win_x64.zip

TASK [show dir contents 2] ***********************************************************************************************************************************************************
task path: /home/jborean/dev/ansible-tester/main.yml:36
changed: [SERVER2022] =>
    changed: true
    cmd: gci C:\temp\test
    delta: '0:00:00.332384'
    end: '2024-01-15 10:28:17.836823'
    rc: 0
    start: '2024-01-15 10:28:17.504439'
    stderr: ''
    stderr_lines: <omitted>
    stdout: |4-


            Directory: C:\temp\test


        Mode                 LastWriteTime         Length Name
        ----                 -------------         ------ ----
        -a----         1/15/2024  10:28 AM           1009 fsagent1915_win_x64.zip
    stdout_lines: <omitted>

PLAY RECAP ***************************************************************************************************************************************************************************
SERVER2022                 : ok=6    changed=6    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0

worshach · 2024-01-17T16:44:51Z

Thank you. That does seem to have worked.

worshach closed this as completed Jan 17, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

win_get_url doesn't get correct filename for URL's containing parameters #575

win_get_url doesn't get correct filename for URL's containing parameters #575

worshach commented Jan 10, 2024 •

edited by jborean93

Loading

jborean93 commented Jan 15, 2024

worshach commented Jan 17, 2024

win_get_url doesn't get correct filename for URL's containing parameters #575

win_get_url doesn't get correct filename for URL's containing parameters #575

Comments

worshach commented Jan 10, 2024 • edited by jborean93 Loading

SUMMARY

ISSUE TYPE

COMPONENT NAME

ANSIBLE VERSION

COLLECTION VERSION

CONFIGURATION

OS / ENVIRONMENT

STEPS TO REPRODUCE

EXPECTED RESULTS

ACTUAL RESULTS

jborean93 commented Jan 15, 2024

worshach commented Jan 17, 2024

worshach commented Jan 10, 2024 •

edited by jborean93

Loading