Un-fork Azure SDK

[arpad-m]

Starting with #10036, we are using a forked version of the Azure SDK so that we only use one async runtime instead of two.

Blockers for un-forking:

• fixing of async-io transitive dependency Azure/azure-sdk-for-rust#1652
• (ideally) implementation of Add support for Blob Batch Azure/azure-sdk-for-rust#1068, cc Azure: proper bulk deletion #7931
• (ideally) fix of Azure Workload Identity - Expired token Azure/azure-sdk-for-rust#1739 so that we can revert fix: force-refresh azure identity token #10378
• release of official SDK from the main branch (we currently use the legacy branch of the SDK).

[heaths]

Note: whether using our legacy branch of what's in main now, you're not tied to a single async runtime or HTTP stack. We default to reqwest and, therefore, tokio, but you can provide your own HttpClient implementation. We've also been careful to limit production code to using abstractions from std and futures - only tests and examples should use tokio - but if you find any problems, please do file an issue.

[arpad-m]

The azure SDK's usage of tokio isn't the issue. We've already been heavy tokio users before, and are still such users now.

The problem is that azure_identity spawns processes using a non-tokio runtime (async-io). I have filed an issue in the SDK repo and we've already talked there, it's the first entry in the blockers list above: Azure/azure-sdk-for-rust#1652 . The process spawning should happen very rarely (in terms of CPU time eaten by the other runtime) and probably only during initialization, but we still would like to avoid the dependency on a non-tokio runtime so that we can cargo-deny it.

So in other words, the fork was made in order to include Azure/azure-sdk-for-rust#1654.

---

A bit unrelated, but as you mention HttpClient, this might interest you: even though we only use tokio, it's really great that we can provide a custom HttpClient implementation, that way we can configure reqwest to our liking.

We used this ability in #10169 to enable reqwest's http connection pooling (option enabled in #10324). So far the reqwest issue that motivated turning off http connection pooling in the SDK didn't manifest for us, but pooling solved an important issue for us.

Namely, before we were running into tcp port exhaustion issues on our servers because we were uploading a lot of files to Azure Blob Storage very quickly, and each upload would mean one new http connection on top of one new tcp stream, both of which would then promptly be closed. The kernel keeps the port of tcp streams allocated for a while after the stream has been closed (TIME_WAIT), to not misattribute any stray incoming packets to the new connection.