Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rust] Rust Storage SDK Wrapper Proof of Concepts #1

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
bd65517
Initial wrapping work, BlobClient with working e2e download, no auth
vincenttran-msft Oct 30, 2024
bc60739
Moved gen code into our dir, rough draft StorageHeadersPolicy, having…
vincenttran-msft Oct 31, 2024
da34a57
Modifiable options POC, next step is bringing in auth and verifying S…
vincenttran-msft Oct 31, 2024
3d8fad6
Added StorageHeadersPolicy to per_call_policies, but weird side effec…
vincenttran-msft Nov 1, 2024
11afbc6
Working authenticated download
vincenttran-msft Nov 2, 2024
c04ebff
Fleshed out more of ContainerClient, BlobServiceClient, added get_blo…
vincenttran-msft Nov 5, 2024
d30397c
Found root-cause of API failures: %-encoding
vincenttran-msft Nov 5, 2024
ba209be
All written tests working except commented, double-ended issues with …
vincenttran-msft Nov 6, 2024
2818038
Finally working tags header value, working options bag
vincenttran-msft Nov 7, 2024
61e55eb
Refactor get-subclient, bring back type-state for blob types(append,p…
vincenttran-msft Nov 12, 2024
0282be7
Attempt to make it one crate
vincenttran-msft Nov 12, 2024
c71e264
Create container working, list blobs and containers working
vincenttran-msft Nov 13, 2024
0d081a6
stage_block and commit_block_list fleshed out, but too many issues to…
vincenttran-msft Nov 14, 2024
39fa6b3
Generated code changes necessary
vincenttran-msft Nov 14, 2024
bf66185
Merge branch 'main' into vincenttran/wrapping_proof_of_concept
vincenttran-msft Nov 14, 2024
906db1c
Working overwrite options bag, example for pub(crate) options in opti…
vincenttran-msft Nov 15, 2024
9b081bc
Working setting tags on upload, sample helper
vincenttran-msft Nov 15, 2024
96d5ca8
Slight cleanup, ranged downloads
vincenttran-msft Nov 19, 2024
e1e96f3
blob_client.rs all passing w/ asserts
vincenttran-msft Nov 19, 2024
a597ebf
Code cleanup
vincenttran-msft Nov 21, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ members = [
"eng/test/mock_transport",
"sdk/storage",
"sdk/storage/azure_storage_blob",
"sdk/storage/blob_storage",
]

[workspace.package]
Expand Down
12 changes: 12 additions & 0 deletions sdk/storage/azure_storage_blob/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,18 @@ rust-version.workspace = true

[dependencies]
azure_storage_common.workspace = true
async-std = { workspace = true }
azure_core = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
time = { workspace = true }
typespec_client_core = { workspace = true, features = ["reqwest"] }
typespec_derive = { workspace = true }
blob_storage = { path = "../blob_storage" }
uuid = { workspace = true }

[dev-dependencies]
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }

[lints]
workspace = true
132 changes: 132 additions & 0 deletions sdk/storage/azure_storage_blob/src/clients/blob_client.rs
Original file line number Diff line number Diff line change
@@ -1,2 +1,134 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use azure_core::{Context, Method, Request, Response, Result, Url};
use blob_storage::blob_blob::BlobBlobDownloadOptions;
use blob_storage::blob_client::BlobClientOptions;
use blob_storage::BlobClient as GeneratedBlobClient;
use std::sync::Arc;
use uuid::Uuid;

// Later this will be auto-populated with current version, otherwise could take user input as well
// const CURRENT_SVC_VERSION: String = String::from("2024-08-04");
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

pub struct BlobClient {
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
account_name: String,
container_name: String,
blob_name: String,
client: GeneratedBlobClient,
}
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

impl BlobClient {
pub fn new(
account_name: String,
container_name: String,
blob_name: String,
options: Option<BlobClientOptions>,
) -> Result<Self> {
// Build Blob URL from input
let endpoint = "https://".to_owned() + &account_name + ".blob.core.windows.net/";
let options = options.unwrap_or_default();
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
let client = GeneratedBlobClient::with_no_credential(endpoint, Some(options))?;

Ok(Self {
account_name: account_name.clone(),
container_name: container_name.clone(),
blob_name: blob_name.clone(),
client: client,
})
}

// fn build_url(account_name: &str) -> String {
// "https://".to_owned() + account_name + ".blob.core.windows.net/"
// }

pub async fn download_blob(
&self,
options: Option<BlobBlobDownloadOptions<'_>>, // Curious if this is the right move, or if we can do a simple wrapper with an easy Into convert to the generated version
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
) -> Result<Response<Vec<u8>>> {
//TODO: Inject version through a pipeline policy
let version = Uuid::new_v4().to_string();
self.client
.get_blob_blob_client()
.download(
self.container_name.clone(),
self.blob_name.clone(),
version,
String::from("2024-08-04"),
Some(BlobBlobDownloadOptions::default()),
)
.await
}

// pub async fn get_blob_properties(&self) -> Result<Response> {
// // Build the get properties request itself
// let mut request = Request::new(self.url.to_owned(), Method::Head); // This is technically cloning
// BlobClient::finalize_request(&mut request);

// // Send the request
// let response = self.pipeline.send(&(Context::new()), &mut request).await?;
// println!("Response headers: {:?}", response);

// // Return the entire response for now
// Ok(response)
// }
}

#[cfg(test)]
mod tests {
use super::*;

#[tokio::test]
async fn test_download_blob() {
let blob_client = BlobClient::new(
String::from("vincenttranpublicac"),
String::from("public"),
String::from("hello.txt"),
Some(BlobClientOptions::default()),
)
.unwrap();
let response = blob_client
.download_blob(Some(BlobBlobDownloadOptions::default()))
.await
.unwrap();
print!("{:?}", response);
print!(
"\n{:?}",
response.into_body().collect_string().await.unwrap()
);
}
}

// #[tokio::test]
// async fn test_get_blob_properties() {
// let credential = DefaultAzureCredentialBuilder::default()
// .build()
// .map(|cred| Arc::new(cred) as Arc<dyn TokenCredential>)
// .expect("Failed to build credential");

// // Create a Blob Client
// let my_blob_client = BlobClient::new(
// String::from("vincenttranstock"),
// String::from("acontainer108f32e8"),
// String::from("hello.txt"),
// credential,
// None,
// );

// // Get response
// let ret = my_blob_client
// .get_blob_properties()
// .await
// .expect("Request failed!");
// let (status_code, headers, _response_body) = ret.deconstruct();

// // Assert equality
// assert_eq!(status_code, azure_core::StatusCode::Ok);
// assert_eq!(
// headers
// .get_str(&HeaderName::from_static("content-length"))
// .expect("Failed getting content-length header"),
// "10"
// )
// }
// }
20 changes: 20 additions & 0 deletions sdk/storage/blob_storage/Cargo.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
[package]
name = "blob_storage"
version = "0.1.0"
authors.workspace = true
edition.workspace = true
license.workspace = true
repository.workspace = true
rust-version.workspace = true

[dependencies]
async-std = { workspace = true }
azure_core = { workspace = true }
serde = { workspace = true }
serde_json = { workspace = true }
time = { workspace = true }
typespec_client_core = { workspace = true, features = ["reqwest"] }
typespec_derive = { workspace = true }

[dev-dependencies]
tokio = { version = "1", features = ["macros", "rt-multi-thread"] }
Loading