"bolt_builtin_content" error when using PCP transport in Bolt v3.30.0

[kruegerkyle95]

Describe the Bug

Based on my understanding, in version 3.30.0 of Bolt, the new parameter "bolt_builtin_content" is passed to the "apply_catalog" task that is internal to Bolt. When using PCP transport, the "apply_catalog" task on the Puppet server comes from the "apply_helpers" module which has not been updated to support this new parameter. I think the identical changes need to be made to that module - https://forge.puppet.com/modules/puppetlabs/apply_helpers/readme

Expected Behavior

Apply function to behave identically to pre-3.30.0 Bolt

[donoghuc]

Thanks for the report.

[LukasAud]

Hi @kruegerkyle95, thanks for reporting this. Can I ask what version of Bolt and, if applicable, PE are you using?

As the description of the module states, puppetlabs-apply_helpers used to be a helper module that allowed PE to consume Bolts 'apply' functions. However, more recent versions of Bolt and PE are capable of natively communicating, rendering apply_helpers unnecessary.

[donoghuc]

However, more recent versions of Bolt and PE are capable of natively communicating, rendering apply_helpers unnecessary.

What do you mean by this?

[mcdonaldseanp]

@donoghuc @LukasAud sorry, the misunderstanding is probably just a lack of thorough explanation on my part:

What Lukas is pointing out is that in PE versions past 2021.7 apply_helpers isn't necessary because PE natively supports the apply function. It's not that they can communicate better, but rather that the integrated bolt functionality in PE supports apply.

[kruegerkyle95]

Correct me if I'm wrong, but my understanding is that apply_helpers is still necessary regardless of PE version when the "apply" function is targeting nodes via PCP. When using PCP, all task code that is executed on target nodes is downloaded from the PCP broker, not sourced from Bolt

[donoghuc]

Yeah, if you are using bolt's PCP transport the apply helpers are required (and as you correctly identified there is a bug with them now). Additionally all module content in the plan you are running must be deployed in PE so that target nodes can download it from puppetserver via the "real" PCP transport. Essentially bolt's PCP is just an API interface to PE, Bolt itself is not actually running the PCP protocol.

Sean's sentiment is that essentially PE can run plans with apply blocks natively so if you want to do that you should just use PE. For most configurations I can think of deploying the modules that power plan runs via bolt's PCP transport are too insecure to make bolt's PCP transport useful at all.

[kruegerkyle95]

We don't use PE to run plans due to the issue described in (I believe this is the right ticket number) PE-28333. RBAC controls in PE plans are not sufficient so we've disabled them and only run plans via Bolt. We almost exclusively use PCP rather than any other available protocol because of the large multi-tenant architecture in our environment that would make other protocols generally impractical

[donoghuc]

I guess i dont understand the security tradeoff there. Seems like with bolt, you get essentially task rbac but you lose for the most part code review controls. With PE plans you maintain code review controls but its harder (without wrapper plans) to control what targets are allowed to be run on. Both choices seem sub optimal to me 🤷