Add possibility to retrieve signed URL from storage

[enyo]

The Storage library does not expose a way to fetch a signed URL at the moment.

To do this in our app, we currently need to deploy a copy of private PEM key that we then use to extract the rsaKey from, and use the RS256Signer to create the signature which looks like this:

GET\n\n\n' + '$expirationTime\n' + '/$bucketName/$storagePath

To do this, we need to import to source dart files from googleapis_auth:

// ignore: implementation_imports
import 'package:googleapis_auth/src/crypto/pem.dart';
// ignore: implementation_imports
import 'package:googleapis_auth/src/crypto/rsa_sign.dart';

The actual code that generates the signature:

final serviceAccount = JSON.decode(config.serviceAccountKey);
final pem = serviceAccount['private_key'];
final googleAccessId = serviceAccount['client_email'];
final rsaKey = keyFromString(pem);
final signer = new RS256Signer(rsaKey);
final signedRequestBytes = signer.sign(UTF8.encode(stringToSign));
final base64EncodedSignedRequest = Uri.encodeQueryComponent(BASE64.encode(signedRequestBytes));

The complete URL then looks like this:

https://storage.googleapis.com/$bucketName/$storagePath?GoogleAccessId=$googleAccessId&Expires=$expirationTime&Signature=$base64EncodedSignedRequest

Apart from the fact, that we would like to avoid deploying our service account key, it would be great if there was a function in Storage that would handle this properly.

Maybe there is also another, easier way to generate these URLs. If so, we are happy about any feedback.

[mkustermann]

Regarding the possibility of signing data: There is in fact a way to avoid having a service account key in the app (at least in the cloud): The IAM service provides an API to sign data. It can be used in Dart via package:googleapis/iam/v1.dart. The default oauth2 token available via the metadata service on Compute Engine has the cloud-platform scope -- though please note: you need to give the AppEngine service account an additional permission (namely iam.serviceAccounts.signBlob and maybe also storage.objects.{create,delete,get,update})

You can see this code as an example, how it can be used: This makes signed cloud storage upload URLs for the pub server.

Though of course, the metadata server is only available on the cloud -- for local testing one simply needs a service account.

(Another option is to store the service account in datastore/cloud-storage)

[hiroshihorie]

Still no official way to get a signed url ? 🥺 I modified @enyo's code to generate v2 signed urls, thank you!

[ykaito21]

is there any way to generate V4 signed urls with this package?

[Gathondu]

I need help with the following code that gives me a SignatureDoesNotMatch error.

Future<String> getWavSignedUrl(
      {required String bucketName, required String file}) async {
    const expiration = 172800; // 48 hours
    final canonicalUri = '$bucketName/$file';
    final dateTimeNow = DateTime.now().toUtc();
    final expirationDateStamp =
        '${dateTimeNow.year}${dateTimeNow.month.toString().padLeft(2, '0')}${dateTimeNow.day.toString().padLeft(2, '0')}';
    final expirationDateTimeStamp =
        '${dateTimeNow.toIso8601String().replaceAll('-', '').replaceAll(':', '').split('.')[0]}Z';
    final clientEmail = _accountCredentials.email;
    final credentialScope = '$expirationDateStamp/us/storage/goog4_request';
    final credential = '$clientEmail/$credentialScope';

    Map<String, String> headers = {};
    headers['host'] = '$bucketName.storage.googleapis.com';
    var sortedHeaderKeys = headers.keys.toList()..sort();
    var canonicalHeaders = '';
    var signedHeaders = '';
    for (var key in sortedHeaderKeys) {
      canonicalHeaders += '$key:${headers[key]?.toLowerCase()}\n';
      signedHeaders += '$key;';
    }
    signedHeaders = signedHeaders.substring(
        0, signedHeaders.length - 1); // remove trailing ';'

    Map<String, dynamic> queryParameters = {};
    queryParameters['X-Goog-Algorithm'] = 'GOOG4-RSA-SHA256';
    queryParameters['X-Goog-Credential'] = credential;
    queryParameters['X-Goog-Date'] = expirationDateTimeStamp;
    queryParameters['X-Goog-Expires'] = expiration;
    queryParameters['X-Goog-SignedHeaders'] = signedHeaders;

    var canonicalQueryString = '';
    var sortedQueryKeys = queryParameters.keys.toList()..sort();
    for (var key in sortedQueryKeys) {
      canonicalQueryString += '$key=${queryParameters[key].toString()}&';
    }
    canonicalQueryString = canonicalQueryString.substring(
        0, canonicalQueryString.length - 1); // remove trailing '&'

    String canonicalRequest = [
      'GET',
      canonicalUri,
      canonicalQueryString,
      canonicalHeaders,
      signedHeaders,
    ].join('\n');

    Digest canonicalRequestHash = sha256.convert(utf8.encode(canonicalRequest));
    String stringToSign = [
      'GOOG4-RSA-SHA256',
      expirationDateTimeStamp,
      credentialScope,
      '$canonicalRequestHash',
    ].join('\n');

    var pem = _accountCredentials.privateKey;
    var rsaKey = keyFromString(pem);
    var signer = RS256Signer(rsaKey);
    List<int> stringToSignList = utf8.encode(stringToSign);
    var signedRequestBytes = signer.sign(stringToSignList);
    var base64EncodedSignedRequest = HEX.encode(signedRequestBytes);
    const schemeAndHost = 'https://storage.googleapis.com/';
    var signedUrl =
        '$schemeAndHost$canonicalUri?$canonicalQueryString&X-Goog-Signature=$base64EncodedSignedRequest';
    return signedUrl;
  }

[bryanoltman]

+1 to this issue – this would be super useful and would allow us to use this package in several places we currently use package:http