SFTPOperator with dynamic user, fails.

[remchuk]

What do you see as an issue?

Hey there, This is my first post here so forgive me if this is not the correct place.

I have a unique need of Airflow, I need to integrate SFTP&SSH Operator with a dynamic client, meaning I need that every trigger will also prompt for a user and a password, This means that I can't actually create a permanent Connection in the UI.

I find that using SSHHook is the correct approch.
I created a task that receives the params, and does:

def create_ssh_hook(**kwargs):
    ti = kwargs["ti"]
    ti.xcom_push(key="ssh_hook_host", value=kwargs["params"]["host"])
    ti.xcom_push(key="ssh_hook_username", value=kwargs["params"]["username"])
    ti.xcom_push(key="ssh_hook_password", value=kwargs["params"]["password"])


create_ssh_hook_task = PythonOperator(
     task_id="create_ssh_hook",
     provide_context=True,
     python_callable=create_ssh_hook,
    op_kwargs=params
)


add_file_to_vm = SFTPOperator(
      task_id='add_script_to_vm',
      sftp_hook=SFTPHook(ssh_hook=SSHHook(remote_host="{{ ti.xcom_pull(key='ssh_hook_host', task_ids='create_ssh_hook') }}",  # I know this is not supported after 4.9.0, but I can't find a different way to do it and this is the version we have.
                                          username="{{ ti.xcom_pull(key='ssh_hook_username', task_ids='create_ssh_hook') }}"
                                          password="{{ ti.xcom_pull(key='ssh_hook_password', task_ids='create_ssh_hook') }}")),
     local_filepath='/a',
     remote_filepath='/tmp/a',
     operation='put',
     create_intermediate_dirs=True,
)

create_ssh_hoot_task >> add_file_to_vm

Now I recieve the following error:
socket.gaierror: [Errno -2] Name or service not known

Now when trying to just ssh from the same place that runs this dag, works just fine, also setting up a proper connection with a static user and a ssh_conn_id also works.

Solving the problem

I think the correct way of working with SSHHook should still be implemented.
I tried every docs file but none specify about dynamic connection.

Anything else

No response

Are you willing to submit PR?

• Yes I am willing to submit a PR!

Code of Conduct

• I agree to follow this project's Code of Conduct

[boring-cyborg[bot]]

Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for approval.

[potiuk]

For those kind of things you should rather use discussions.

You should likely use task flow tasks where you instantiate SSHHook and simply use the low level methods of the hook directly - rather than usiing SSH Operator - that would be my advice. Then you do no have to use JINJA and have more control over what and when you pass to the SSH Hook.