Build fails when triggered by scheduler if using Secret Manager

[lisovyk]

I have a custom docker image that runs R script and reads env variables with Sys.getenv().
I have added those variables to Secrets Manager, and am using such code to create a build:

my_r_script <- "script.R"
bs <- cr_buildstep_r(
    my_r_script,
    name = "eu.gcr.io/my_project_id/my_docker_image",
    r_source = 'runtime',
    secretEnv = c('USER', 'PASS'))

my_build <- cr_build_yaml(steps = bs, 
                          timeout = 2400,              
                          availableSecrets = list(cr_build_yaml_secrets('USER', 'USER'),
                                                  cr_build_yaml_secrets('PASS', 'PASS')))
built <- cr_build(my_build)

.. which runs successfully by default cloudbuild account.

When scheduled with cr_schedule("test-script", schedule = "15 5 * * *", httpTarget = cr_schedule_http(built)), the build fails on trigger and returns an error on the line where the env variables are read.

Please, help me resolve this! I've tried to add a custom service account to thecr_build_yaml and cr_build, the latter returns a weird http 400 error:

http_400 generic::invalid_argument: Failed to parse resource name my_service_acc_name@my_project_id.iam.gserviceaccount.com

This google doc gives another way to address service accounts in serviceAccount variable:
'projects/PROJECT_ID/serviceAccounts/SERVICE_ACCOUNT'
... which is not an email as per R function cr_build()

[MarkEdmondson1234]

How are you adding the custom service account? The error indicates the format is wrong, perhaps you need to put it within a list()?

Are you giving your custom service account Secret Manager permissions so it can read the Env Vars?

[lisovyk]

After debugging googleCloudRunner function I managed to run the build with custom service account.
( I've played with the request body that is passed to doHttrRequest())
For serviceAccount I've passed OAuth 2 Client ID, in the form "projects/PROJECT_ID/serviceAccounts/OAuth2ClientID":

built <- cr_build(my_build, serviceAccount = "projects/PROJECT_ID/serviceAccounts/OAuth2ClientID")

OAuth2ClientID can be found on your service account's details page as "Unique ID"
With that, the build successfully ran with needed serviceAccount (presumably? In execution details on the web it shows the ID of account instead of default runner email now//.)

Although, if the build is triggered from cron it still fails for some reason.

Will update if solution is found.

How are you adding the custom service account? The error indicates the format is wrong, perhaps you need to put it within a list()?

I have been passing it as a regular string.
If I put it in a list like this, either in cr_build_yaml() or cr_build():

built <- cr_build(my_build, serviceAccount = list('accName@project_id.iam.gserviceaccount.com'))

I get such error:

http_400 Invalid JSON payload received. Unknown name "serviceAccount" at 'build': Proto field is not repeating, cannot start list.

Seems like a json parsing error..

Are you giving your custom service account Secret Manager permissions so it can read the Env Vars?

Yes, sure. That service account has:
Cloud Build Service Agent
Cloud Run Admin
Cloud Scheduler Admin
Logs Writer
Secret Manager Secret Accessor
Service Account User
Storage Admin

Could you share some example of serviceAccount var usage?

[lisovyk]

I have created a pubsub trigger and it ran successfully on custom account, accessing Secret Manager variables. So probably will have to use this approach for now.

Working code for anyone stumbling here:

my_r_script <- "/script/path/script.R"
bs <- cr_buildstep_r(
    my_r_script,
    name = "eu.gcr.io/PROJECT_ID/image:tag",
    r_source = 'runtime',
    secretEnv = c('VAR1', 'VAR2'))

my_build <- cr_build_yaml(steps = c(bs), 
                          timeout = 2400,
                          availableSecrets = list(cr_build_yaml_secrets('VAR1', 'VAR1'),
                                                  cr_build_yaml_secrets('VAR2', 'VAR2')),
                          options = list('logging' = 'CLOUD_LOGGING_ONLY'),
                          serviceAccount = "projects/PROJECT_D/serviceAccounts/SERVICE_ACCOUNT_UNIQUE_ID"
)
library(googlePubsubR)
pubsub_auth()
topics_create("test-topic")
pubsub_trigger <- cr_buildtrigger_pubsub("test-topic")

bb <- cr_build_make(my_build)
cr_buildtrigger(bb, name = "pubsub-triggered", trigger = pubsub_trigger)
cr_schedule("cloud-build-pubsub",
            "15 5 * * *",
            pubsubTarget = cr_schedule_pubsub("test-topic"))