Support more credential sources in dlt CLI

[trymzet]

Feature description

Currently, the CLI only works when specifying credentials in 3 out of 6 possible ways:

• secrets.toml
• config.toml
• environment variables
• custom config providers
• parameters specified in the pipeline
  For example,

  pipeline = dlt.pipeline(..., destination=dlt.destinations.clickhouse(credentials=my_credentials))

• dynamically provided credentials
  For example, dlt.secrets["my-secret"] = "my_dynamic_secret_value"

In practical terms, this makes eg. using a secret manager for storing credentials troublesome, because users need to have one of the 3 static credential sources available locally, which basically defeats the purpose of using an external secret manager.

Are you a dlt user?

None

Use case

No response

Proposed solution

Probably creds passed via the credentials param should be prioritized, as some of the tutorials use this method of providing credentials (eg. https://dlthub.com/docs/pipelines/sql_database_mssql/load-data-with-python-from-sql_database_mssql-to-mssql#5-next-steps-to-get-your-pipeline-running-in-production), which means if you followed the tutorial, you would end up unable to use dlt CLI commands.

Related issues

No response