Replies: 3 comments 4 replies
-
|
This is still happening with Deno v1.40.4 |
Beta Was this translation helpful? Give feedback.
-
|
just pass s3.putObject(
{
Bucket: Deno.env.get("AWS_S3_BUCKET"),
Key: "test.mp4",
ContentLength: Number(response.headers.get("content-length")),
Body: response.body,
}, |
Beta Was this translation helpful? Give feedback.
-
|
I tried passing the Deno fetch response body directly to S3 putObject but depending on the imports I get two very different errors. If the import is if the import is However, what's really interesting, is if I use this custom S3 request handler it works with either import using exactly the same credentials. import { S3, PutObjectCommandOutput } from "https://esm.sh/@aws-sdk/[email protected]"
// import { S3, PutObjectCommandOutput } from "npm:@aws-sdk/[email protected]"
import { AWSNodeCompatibleRequestHandler } from "https://cdn.jsdelivr.net/gh/tazmaniax/common-code-parts@feat/update/AWSNodeCompatibleRequestHandler/mod.ts"
const denoRequestHandler = new AWSNodeCompatibleRequestHandler()
const awsCredentials = {
accessKeyId: Deno.env.get('AWS_ACCESS_KEY_ID') || '',
secretAccessKey: Deno.env.get('AWS_SECRET_ACCESS_KEY') ?? '',
sessionToken: Deno.env.get('AWS_SESSION_TOKEN') ?? ''
}
const s3 = new S3({
region: Deno.env.get('AWS_REGION') ?? '',
credentials: awsCredentials,
requestHandler: denoRequestHandler
})
const sourceUrl = 'https://download.samplelib.com/mp4/sample-5s.mp4'
const response = await fetch(sourceUrl, {
headers: { Accept: "video/mp4" }
})
if (response.status !== 200) throw new Error(`Failed to fetch '${sourceUrl}' with error '${response.statusText}'`)
if (!response.body) throw new Error(`Undefined response body`)
console.log('Uploading file to S3')
s3.putObject(
{
Bucket: Deno.env.get('AWS_TRANSCRIBE_INPUT_BUCKET'),
Key: 'test.mp4',
ContentLength: Number(response.headers.get("content-length")),
Body: response.body
},
function (error: any, data?: PutObjectCommandOutput | undefined) {
if (error) {
console.error(`Error uploading file to s3`, error)
} else {
console.info(`Success uploading file to s3`, data)
}
}
)I was keen to avoid this custom request handler if possible so why I thought maybe using Node stream |
Beta Was this translation helpful? Give feedback.
-
|
I got this error import { S3Client } from "https://esm.sh/@aws-sdk/[email protected]";
import { Upload } from "https://esm.sh/@aws-sdk/[email protected]";
const s3 = new S3Client({
region: "us-east-1",
endpoint: "...redacted", // specify minio endpoint
credentials: {
accessKeyId: "...redacted",
secretAccessKey: "...redacted",
},
});
async function uploadBinary() {
const sourceUrl = "http://localhost:3000/foobar.mkv";
const response = await fetch(sourceUrl);
if (response.status !== 200) {
throw new Error(
`Failed to fetch '${sourceUrl}' with error '${response.statusText}'`,
);
}
if (!response.body) throw new Error(`Undefined response body`);
const parallelUploads3 = new Upload({
client: s3,
params: {
Bucket: "videos",
Key: "video.mkv",
Body: response.body,
},
});
parallelUploads3.on("httpUploadProgress", (progress) => {
console.log(progress);
});
await parallelUploads3.done();
}
await uploadBinary(); |
Beta Was this translation helpful? Give feedback.
-
|
Are you aware of this little deno library? I've had good luck using it https://deno.land/x/[email protected] |
Beta Was this translation helpful? Give feedback.
-
|
@jtoppine thanks. I have heard of it but I'm using several AWS services and just want to use one library if possible. For S3, I need to be able to stream data from another source and there wasn't an example of that. Also concerned that this project hasn't been updated in 3 years and has a big warning "This project is work in progress. Expect breaking changes." |
Beta Was this translation helpful? Give feedback.
-
|
https://jsr.io/@bradenmacdonald/s3-lite-client is maintained (by me), works with Deno, and supports streaming - in case that's helpful. Doesn't work with other AWS services though. |
Beta Was this translation helpful? Give feedback.
-
I'm trying to stream a file to S3 and for that I need a Node Readable to pass into S3 putObject() but I'm getting the following compile error at
Readable.fromWeb(response.body)using Deno 1.40.2. I feel this should work but maybe there is a better approach. ThanksBeta Was this translation helpful? Give feedback.
All reactions