Support pool.sync_updates from remote_pool repo

[gangj]

CP-50787 CP-51347: Support pool.sync_updates from remote_pool repo

When a remote_pool type repository, which points to the enabled
repository in the remote pool coordinator, is set as the enabled
repository of the pool, updates can be synced from it with API
pool.sync_updates.

The username password of the remote pool coordinator is required as
parameters for pool.sync_updates to login the remote pool.

And the remote pool coordinator's host server certificate needs to be
configured in the remote_pool repository, it will be used to verify the
remote end when sending out username passwords.

A new yum/dnf plugin "xapitoken" is introduced to set xapi token as HTTP
cookie: "session_id" for each HTTP request which downloads files from the
remote_pool repository.

CP-52245: Temp disable repo_gpgcheck when syncing from remote_pool repo

Will re-enable repo_gpgcheck after CP-51429 is done by reverting this
commit.

[gangj]

Now when the certificate is invalid, the expected ssl_verify_error is not raised, instead this Xmlrpc_client.Connection_reset is caught here. I think this problem can be resolved with another ticket later.

[minglumlu]

I think the context of the error is clear as it would happen when a pool.sync_updates. It could be as simple as "CANNOT_CONTACT_HOST".
But the authentication (incorrect password) may fail with this error also. So the possible causes should be mentioned in the message.

[gangj]

The authentication (incorrect password) failure will fail with another error: update_syncing_remote_pool_coordinator_service_failed.
And we have a corresponding error pool_joining_host_connection_failed.
If we change it to a general one, I think we will need to provide more info like the address and port of the host which failed to be contacted, like the other existing one: error Api_errors.tls_connection_failed ["address"; "port"]

[minglumlu]

To be useful to the user, how about to use "HOST_OFFLINE"?

[gangj]

This error corresponds to the error during pool join: pool_joining_host_service_failed, I think it is not host offline, it is the HTTP service failed to respond.

[minglumlu]

The location of the comment looks odd. It is auto-formatted like this?

[gangj]

I added there and it had been formatted, or is there a better suggested place to add the comment?

[minglumlu]

I think it's too early exposing the localhost here.

[gangj]

It is used to setup the binary_url which is pointing to the endpoint proxied by stunnel, so it is the localhost.

[minglumlu]

The binary_url is determined by the ServerAuthConf. It should be determined by the handling of ServerAuthConf.

[minglumlu]

Separating the function config_repo would look better:

let proxy_params =
  let proxy_url_param, proxy_username_param, proxy_password_param =
    get_proxy_params ~__context repo_name
  in
  [
  ; proxy_url_param
  ; proxy_username_param
  ; proxy_password_param
  ]
in

let config_repo params =
  let Pkg_mgr.{cmd; params'} =
    "--save" :: params
    |> fun config -> Pkgs.config_repo ~repo_name ~config
  in
  ignore
    (Helpers.call_script ~log_output:Helpers.On_failure cmd params')
in

And with the above, it can be like:

config_repo (List.rev_append proxy_params auth_params)

[minglumlu]

Would suggest keeping consistency with with_sync_client_auth. E.g. with_sync_server_auth.

[minglumlu]

To be clear, it could be client_auth.

[minglumlu]

And If the login_with_password bit is handled before constructing PoolExtHostAuthConf, it will not be necessary to map PoolExtHostAuthConf to ExtHostAuth.

[minglumlu]

This part can be generalized as a function being like Helpers.login_after_server_check.

[minglumlu]

This function can be like:

let with_sync_client_auth auth f =
  let go_with_client_plugin cred plugin_name =
    let ( let@ ) g x = g x in
    let@ temp_file =
      Helpers.with_temp_file_of_content ~mode:[Open_text] "token-" ".json"
        (Yojson.Basic.to_string cred)
    in
    f [temp_file]
  in
  match auth with
  | NoClientAuth ->
      f []
  | ExtHostAuth xapi_token, plugin ->
      let cred = `Assoc [("xapitoken", `String xapi_token)] in
      go_with_client_plugin cred plugin
  | CdnTokenAuth (token_id, token), plugin ->
      (* The validity of the token/token_id can be verified at the beginning of the API call. *)
      let cred = `Assoc [("token", `String token); ("token_id", `String token_id)] in
      go_with_client_plugin cred plugin

[minglumlu]

If both the token_id and token are empty string, it could return NoAuth here. This avoids the empty token_path and checking against it later.
And the inconsistency like non-empty token_id and empty token will result in other error (like auth failure) raised from the CDN server side. This is expected. So we could rely on that instead of checking it in xapi. This will make the with_sync_client_auth and generating plugin parameter simpler as it will not be necessary to handle something like:

match token_path with
| "" ->
    ""

and

  match auth with
  | CdnTokenAuth {token_id; token}, plugin when token_id = "" && token = "" ->
      f (Some ("", plugin))
  | CdnTokenAuth {token_id; token}, plugin when token_id <> "" && token <> "" ->

[minglumlu]

I think the plugin name should be part of the type. This can avoid the empty plugin name for NoAuth following in line 171:

(Uri.to_string uri, None, true, (NoAuth, ""), NoAuth)

[minglumlu]

The config_repo is working as accept a client auth tuple and add proxy under the hood.
Instead, the yum_conf can be just a prepared sync client params, e.g. with name client_auth_params. And proxy can be another sync client params prepared outside of the config_repo, e.g. with name client_proxy_params. Then the config_repo is just to accept these params to run the command with all the parameters.