Proposal: build variant managed by boostrap package

[rocodes]

Proposal:

Proposal with more detail on two separate rpm boostrap packages, securedrop-workstation-keyring{,staging} and how it will help us with cleaner encapsulation and separation of prod from non-prod content/logic.

After #1058 (comment) I realized I needed to write stuff up more formally and it probably belongs in a separate issue/proposal instead of a ticket comment.

Affected components

SecureDrop Workstation

People and roles

sd maintainers

Problem Statement

A few related problems:

• Workstation rpm ships with some non-prod artifacts (test key, yaml that includes prod and staging repos), as well as non prod logic (salt logic that includes conditional provisioning depending on dev, staging, prod) Protect production users from setting up a dev or staging environment in error using production RPM #1058 Avoid manual downloading of dom0 rpm during bootstrapping #945
• Branching salt logic is error prone/hard to read/slows down provisioning (various)
• Conditional logic in systemd services relying on ConditionPathExists and flag files/directories (/var/lib/securedrop-workstation/{dev,prod,staging}) is a bit messy and potentially brittle Use preset files for conditionally enabling/disabling systemd services #1052

Solution impact

• don't ship non-prod stuff in prod packages
• easier to reason about your system
• reduce time on salt runs

Requirements and constraints

• Previous requirement/constraint: "don't want to maintain two different dom0 config RPMs" (this is reasonable, this was a comment from Kev when I suggested having a separate dev vs prod rpm for similar reasons). Constraint: securedrop-workstation-dom0-config is one rpm.
• Prod by default: constraint should be things should "fail closed/fail to prod configuration" (don't have a situation where a prod user could accidentally not have a service or system enabled; eg logind overrides).

Initial proposal

My proposal is as follows:

• securedrop-workstation-dom0-config will depend on a boostrap package that provides repo config and signing key config (already decided, Avoid manual downloading of dom0 rpm during bootstrapping #945 ) and ideally, a apt-sources file as well.
• securedrop-workstation-dom0-config stops shipping any non-prod logic. (More on this later - it's readily doable). With the repo config out of the way, we also remove any conditional logic that includes mentions of staging and dev (such as conditionally-enabled systemd services and flag files written to /var/lib/securedrop-workstation).
  • systemd services will be configured via a preset file that is shipped in the dom0 rpm. (Random minor detail? totally. but relevant in a minute Use preset files for conditionally enabling/disabling systemd services #1052).
• the prod boostrap package is securedrop-workstation-keyring. An alternative staging/test boostrap package available to devs and hosted on yum-test provides the dev bootstrap (yum-test .repo file, test key, apt-test sources file). And, it also provides any additional static configuration files needed to "de-prod-ify" the prod setup.
  • Specifically: systemd preset files and .conf files with a lower # (so they take precedence) that disable unwanted prod changes, like our power management/login settings
  • This makes it explicit that the dev/non-prod setup is a "spin" on top of the prod setup, but the prod setup is the only one that is canonically shipped.
  • All of this could be contained in the workstation-keyring spec file, and the repo would still have under 10 static files in total.

Key points:

• Known states (prod, dev, staging) as opposed to potentially mixed states or user error via config.json clashing with what rpm is installed, for example
• non-prod artifacts/logic are encapsulated and never reach prod systems
• removing branching logic in Salt
• The only things permitted in the boostrap repos are static files (not introducing another area of conditional logic, python code, etc)

Selected proposal

[zenmonkeykstop]

The use of systemd presets with priorities definitely addresses one concern with this approach in terms RPMs managing the same config files. How would the yum-test and apt-test configs get applied to override the prod versions though?

[rocodes]

How would the yum-test and apt-test configs get applied to override the prod versions though?

For the non repo related files (eg systemd presets, conf overrides) it's straightforward: if our systemd presets in prod are 95-securedrop-systemd-user.preset and 95-securedrop-systemd.preset for user units and system units respectively, the dev/staging package would ship a different preset file with a lower number (94-securedrop-systemd.preset) that would take precedence without interfering with the prod files.

For the repo and signing key, the tldr is "we could decide how we want it to work / it could use package-based or file-based constraints that we specify in the rpm .spec file(s)."

Say we ship prod package securedrop-workstation-keyring-0.1.0. A staging package would have a newer version number (securedrop-workstation-keyring-staging-0.2.0-rc1) and we could decide exactly what the constraint relationship could be. At one extreme, we could declare that the packages Conflict, and require the user to uninstall the prod keyring before reinstalling the staging setup. At the other extreme, we could allow the packages to exist side by side, with the higher version number (dev/staging) taking precedence. This option is (only slightly) more complicated, because it would mean we'd be permitting both the staging and prod key and repo files to be on the dev machine, and we'd need to either tell the dom0 config rpm which repo file to prioritize, or allow the dev package to overwrite the repo files even if the two packages don't conflict. But I see this as something that could be solved in a few different ways depending on our desired behaviour.

[rocodes]

In advance of a conversation about this, another related issue about managing dev/staging/prod config: #1053
(relevant because for example we could have a pillar of (apt) repo config data, which I think we should do, and we could have dev or staging-only logic to switch that over to whatever the desired configuration is).

The main thing I am hoping for is that we consolidate around only shipping prod configurations in the dom0 config rpm and the prod keyring rpm, but I am flexible on how we accomplish this plus maintain developer convenience, and I think there are a number of ways we could do that.

[rocodes]

Discussed today in meeting:

• None of this (except the prod keyring package as already agreed upon) is a blocker or requirement for 1.1.0.
  • There will be slight changes to the salt logic in preparation for that, since the prod package will manage the repo files for prod, but there won't be an equivalent for dev/staging.
• Broad support for removing conditional (if environment != "prod") etc stuff from salt runs. @rocodes to propose a few lightweight paths forward.
• For now (and 1.1.0) we'll focus on yum .repo file and key in the boostrap repo, nothing else. can revisit the concept of apt-sources in there later on, but it does conceptually break some of our "what is locate where" assumptions, so it might not be the best fit.
• There's a distinction between building a package and using packaging logic to manage files, vs deploying the package - so we could consider a locally-built dev/staging rpm as part of the make dev / make staging targets we have now, without necessarily planning to put a staging keyring package on apt-test. For now, no plan to publish a staging keyring package.
• A few other things we discussed: possibility of a dev/staging make target that manages the files we want; possibility of just refactoring the branching logic in salt as a first step so it's all in one place (and or a hybrid approach).

Still to be resolved:

• Should the workstation-dom0-config Depend on the workstation-keyring package?
• How to manage apt-sources (maybe defaulting to prod in the dom0 config rpm is a good idea; there was also a mention of build conditions with different files in an rpm, which is possible Use conditional build in rpm to manage dev-only settings #1031 , but we should not put anything in the securedrop-workstation-keyring rpm that does not belong on prod, so this would have to happen in the workstation repo.)