Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Rust] Rust Storage SDK Wrapper Proof of Concepts #1

Draft
wants to merge 20 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
bd65517
Initial wrapping work, BlobClient with working e2e download, no auth
vincenttran-msft Oct 30, 2024
bc60739
Moved gen code into our dir, rough draft StorageHeadersPolicy, having…
vincenttran-msft Oct 31, 2024
da34a57
Modifiable options POC, next step is bringing in auth and verifying S…
vincenttran-msft Oct 31, 2024
3d8fad6
Added StorageHeadersPolicy to per_call_policies, but weird side effec…
vincenttran-msft Nov 1, 2024
11afbc6
Working authenticated download
vincenttran-msft Nov 2, 2024
c04ebff
Fleshed out more of ContainerClient, BlobServiceClient, added get_blo…
vincenttran-msft Nov 5, 2024
d30397c
Found root-cause of API failures: %-encoding
vincenttran-msft Nov 5, 2024
ba209be
All written tests working except commented, double-ended issues with …
vincenttran-msft Nov 6, 2024
2818038
Finally working tags header value, working options bag
vincenttran-msft Nov 7, 2024
61e55eb
Refactor get-subclient, bring back type-state for blob types(append,p…
vincenttran-msft Nov 12, 2024
0282be7
Attempt to make it one crate
vincenttran-msft Nov 12, 2024
c71e264
Create container working, list blobs and containers working
vincenttran-msft Nov 13, 2024
0d081a6
stage_block and commit_block_list fleshed out, but too many issues to…
vincenttran-msft Nov 14, 2024
39fa6b3
Generated code changes necessary
vincenttran-msft Nov 14, 2024
bf66185
Merge branch 'main' into vincenttran/wrapping_proof_of_concept
vincenttran-msft Nov 14, 2024
906db1c
Working overwrite options bag, example for pub(crate) options in opti…
vincenttran-msft Nov 15, 2024
9b081bc
Working setting tags on upload, sample helper
vincenttran-msft Nov 15, 2024
96d5ca8
Slight cleanup, ranged downloads
vincenttran-msft Nov 19, 2024
e1e96f3
blob_client.rs all passing w/ asserts
vincenttran-msft Nov 19, 2024
a597ebf
Code cleanup
vincenttran-msft Nov 21, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

99 changes: 44 additions & 55 deletions sdk/storage/azure_storage_blob/src/clients/blob_client.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use crate::clients::units::*;
use crate::policies::storage_headers_policy::StorageHeadersPolicy;
use azure_core::credentials::TokenCredential;
use azure_core::headers::HeaderName;
Expand All @@ -12,18 +13,20 @@ use azure_identity::DefaultAzureCredentialBuilder;
use blob_storage::blob_blob::{BlobBlobDownloadOptions, BlobBlobGetPropertiesOptions};
use blob_storage::blob_client::BlobClientOptions;
use blob_storage::BlobClient as GeneratedBlobClient;
use std::marker::PhantomData;
use std::sync::Arc;
use uuid::Uuid;

pub struct BlobClient {
endpoint: String,
container_name: String,
blob_name: String,
credential: Option<Arc<dyn TokenCredential>>,
client: GeneratedBlobClient,
pub struct BlobClient<BlobType = Unset> {
pub(crate) blob_type: PhantomData<BlobType>,
pub(crate) endpoint: String,
pub(crate) container_name: String,
pub(crate) blob_name: String,
pub(crate) credential: Option<Arc<dyn TokenCredential>>,
pub(crate) client: GeneratedBlobClient,
}
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved

impl BlobClient {
impl BlobClient<Unset> {
const VERSION_ID: &'static str = ("2024-08-04");

pub fn new(
Expand Down Expand Up @@ -60,6 +63,7 @@ impl BlobClient {
GeneratedBlobClient::with_no_credential(endpoint.clone(), Some(options.clone()))?;

Ok(Self {
blob_type: PhantomData::<Unset>,
endpoint: endpoint.clone(),
container_name: container_name.clone(),
blob_name: blob_name.clone(),
Expand All @@ -68,6 +72,20 @@ impl BlobClient {
})
}

pub async fn as_append_blob(&self) -> BlobClient<Append> {
BlobClient {
blob_type: PhantomData::<Append>,
endpoint: self.endpoint.clone(),
container_name: self.container_name.clone(),
blob_name: self.blob_name.clone(),
credential: self.credential.clone(),
client: GeneratedBlobClient {
endpoint: self.client.endpoint.clone(),
pipeline: self.client.pipeline.clone(),
},
}
}

pub async fn download_blob(
&self,
options: Option<BlobBlobDownloadOptions<'_>>,
Expand Down Expand Up @@ -103,21 +121,12 @@ impl BlobClient {
)
.await
}
}

// pub fn get_container_client(&self) ->

// pub async fn get_blob_properties(&self) -> Result<Response> {
// // Build the get properties request itself
// let mut request = Request::new(self.url.to_owned(), Method::Head); // This is technically cloning
// BlobClient::finalize_request(&mut request);

// // Send the request
// let response = self.pipeline.send(&(Context::new()), &mut request).await?;
// println!("Response headers: {:?}", response);

// // Return the entire response for now
// Ok(response)
// }
impl<Append> BlobClient<Append> {
pub async fn append_block(&self) {
todo!()
}
}

#[cfg(test)]
Expand Down Expand Up @@ -216,38 +225,18 @@ mod tests {
response.into_body().collect_string().await.unwrap()
);
}
}

// #[tokio::test]
// async fn test_get_blob_properties() {
// let credential = DefaultAzureCredentialBuilder::default()
// .build()
// .map(|cred| Arc::new(cred) as Arc<dyn TokenCredential>)
// .expect("Failed to build credential");

// // Create a Blob Client
// let my_blob_client = BlobClient::new(
// String::from("vincenttranstock"),
// String::from("acontainer108f32e8"),
// String::from("hello.txt"),
// credential,
// None,
// );

// // Get response
// let ret = my_blob_client
// .get_blob_properties()
// .await
// .expect("Request failed!");
// let (status_code, headers, _response_body) = ret.deconstruct();

// // Assert equality
// assert_eq!(status_code, azure_core::StatusCode::Ok);
// assert_eq!(
// headers
// .get_str(&HeaderName::from_static("content-length"))
// .expect("Failed getting content-length header"),
// "10"
// )
// }
// }
#[tokio::test]
async fn test_get_append_client() {
let blob_client = BlobClient::new(
String::from("https://vincenttranpublicac.blob.core.windows.net/"),
String::from("public"),
String::from("hello.txt"),
None,
Some(BlobClientOptions::default()),
)
.unwrap();
let append_block_client = blob_client.as_append_blob().await;
append_block_client.append_block();
}
}
21 changes: 13 additions & 8 deletions sdk/storage/azure_storage_blob/src/clients/blob_service_client.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT License.

use crate::clients::units::*;
use crate::policies::storage_headers_policy::StorageHeadersPolicy;
use azure_core::credentials::TokenCredential;
use azure_core::headers::HeaderName;
Expand All @@ -18,6 +19,7 @@ use blob_storage::blob_container::{
use blob_storage::blob_service::BlobServiceGetPropertiesOptions;
use blob_storage::models::StorageServiceProperties;
use blob_storage::BlobClient as GeneratedBlobClient;
use std::marker::PhantomData;
use std::sync::Arc;
use uuid::Uuid;

Expand Down Expand Up @@ -88,14 +90,17 @@ impl BlobServiceClient {
blob_name: String,
options: Option<BlobClientOptions>,
) -> BlobClient {
BlobClient::new(
self.endpoint.clone(),
container_name.clone(),
blob_name.clone(),
self.credential.clone(),
options,
)
.unwrap()
BlobClient {
blob_type: PhantomData::<Unset>,
endpoint: self.client.endpoint.clone().to_string(),
container_name: container_name,
blob_name: blob_name,
credential: self.credential.clone(),
client: GeneratedBlobClient {
endpoint: self.client.endpoint.clone(),
pipeline: self.client.pipeline.clone(),
vincenttran-msft marked this conversation as resolved.
Show resolved Hide resolved
},
}
}
}
#[cfg(test)]
Expand Down
1 change: 1 addition & 0 deletions sdk/storage/azure_storage_blob/src/clients/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@
mod blob_client;
mod blob_container_client;
mod blob_service_client;
mod units;
9 changes: 9 additions & 0 deletions sdk/storage/azure_storage_blob/src/clients/units.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
pub trait BlobKind {}
impl BlobKind for Unset {}
impl BlobKind for Block {}
impl BlobKind for Page {}
impl BlobKind for Append {}
pub struct Unset;
pub struct Block;
pub struct Page;
pub struct Append;