Skip to content

2.1.0 Test Plan

Cory Francis Myers edited this page Oct 15, 2021 · 10 revisions

2.1.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report.

If you have submitted a QA report already for a 2.1.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the release-specific changes as well as changes since the previous release candidate.

If you are testing the upgrade scenario, you should create source and journalist accounts before performing the upgrade - some test cases require existing accounts.

In order to test #5988 you will need to enable HTTPS on the source interface. To do so using self-signed certs:

  • Use the command cd ~/Persistent/securedrop && make self-signed-https-certs on the Admin Workstation to generate the necessary files
  • Choose the appropriate options while running ./securedrop-admin sdconfig
  • Accept the self-signed cert when prompted with the HTTPS warning in Tor Browser.

Environment

  • Install target:
  • Tails version:
  • Test Scenario:
  • SSH over Tor:
  • Release candidate:
  • General notes:

Basic Server Testing

  • After installing the testinfra dependencies, all tests in ./securedrop-admin verify are passing:
    • Install dependencies on Admin Workstation with cd ~/Persistent/securedrop && ./securedrop-admin setup -t
    • Run tests with ./securedrop-admin verify (this will take a while)
    • Remove test dependencies: rm -rf admin/.venv3/ && ./securedrop-admin setup
  • QA Matrix checks pass

Command Line User Generation

  • Can successfully add admin user and login

Administration

  • I have backed up and successfully restored the app server following the backup documentation
  • If doing upgrade testing, make a backup on 2.0.2 and restore this backup on this release candidate
  • "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent
  • Can successfully add journalist account with HOTP authentication

Application Acceptance Testing

Source Interface

Landing page base cases
  • JS warning bar does not appear when using Security Slider high
  • JS warning bar does appear when using Security Slider Low
First submission base cases
  • On generate page, refreshing codename produces a new 7-word codename
  • On submit page, empty submissions produce flashed message
  • On submit page, short message submitted successfully
  • On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
  • On submit page, file less than 500 MB submitted successfully
Returning source base cases
  • Nonexistent codename cannot log in
  • Empty codename cannot log in
  • Legitimate codename can log in
  • Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases
  • Can log in with 2FA tokens
  • incorrect password cannot log in
  • invalid 2fa token cannot log in
  • 2fa immediate reuse cannot log in
  • Journalist account with HOTP can log in
Index base cases
  • Filter by codename works
  • Starring and unstarring works
  • Click select all selects all submissions
  • Selecting all and clicking "Download" works
Individual source page
  • You can submit a reply and a flashed message and new row appears
  • You cannot submit an empty reply
  • Clicking "Delete Source Account" and the source and docs are deleted
  • You can click on a document and successfully decrypt using application private key

Basic Tails Testing

After updating to this release candidate and running securedrop-admin tailsconfig

  • The Updater GUI appears on boot
  • Updating to 2.0.2 is successful

2.1.0 release-specific changes

Web Applications

  • #6075 - valid HTML time tags

    • Verify that the SI lookup page HTML validates correctly (e.g. by using a Firefox addon)
  • #5695 - scrypt and sessions refactor

    • Verify that no codename- or session-related errors were observed during application acceptance testing
    • On the Source Interface, create a new user, log out, return to the home page without clearing the Tor Browser identity, and repeat this process at least 5 times. Verify that you do not observe an error wwith the text "you have been logged out due to inactivity" while repeating these steps.
    • If possible, log in to the Source Interface simultaneously as separate users from multiple instances of the Tor Browser, submit messages from each user, and reply to the users from the Journalist Interface.
      • Verify that each user has a distinct codename in the SI and journalist designation in the JI
      • Verify that no errors were observed on the Source Interface in any user session
      • Verify that submitted messages appear on the correct source page on the JI
      • Verify that sources receive the correct reply in the SI and that the reply can be read
      • On the application server, verify that each source has a distinct GPG key fingerprint, e.g. with the command sudo -u www-data gpg --homedir /var/lib/securedrop/keys -k
    • (Upgrade-only) Verify that a source user created before the upgrade can still log in using their existing codename, that their journalist designation is unchanged, and that they can still read existing and new replies
  • #6041 - SI a11y refactor

    1. Option 1, low effort: review automated accessibility checks
      • Verify that Source Interface functionality and appearance is unchanged
      • Using the Web Developer tools' Accessibility Inspector and check for issues: All, verify that accessibility issues have been significantly reduced on the SI /lookup page vs the corresponding page on https://demo-source.securedrop.org
      • Compare other pages on the SI in the same way - verify that issues have been removed and no new issues introduced.
    2. Option 2, medium effort: step through submission flow in screen reader
      • On Tails, activate the screen reader from the accessibility menu
      • Step through the submission flow at https://demo-source.securedrop.org and pay attention to how the experience feels
      • Step through the submission flow in your QA instance and compare how the experience feels
  • #5958 - OTP secret length

    • verify that the TOTP secret and corresponding QRcode displayed via manage.py add-admin on the application server are 32 chars long and can be used to generate valid TOTP codes via the Google Authenticator mobile app
    • verify that the TOTP secret and QRCode generated via the JI during user creation are also 32 chars long and valid
    • (upgrade-only) verify that an existing 16-char shared secret can still be used to generate valid TOTP codes and log in
    • (upgrade-only) verify that when a user account with an existing 16-char shared secret has its OTP secret regenerated, the new OTP shared secret is 32 chars long and valid.
    • (HOTP-only) verify that an account using HOTP can still be created and used to log in.
  • #5696 - no JS in user delete modal

    • Set the Tor Browser security level to Safest
    • In the Journalist Interface, verify that a confirmation modal dialog is displayed when a user is selected for deletion
    • Choose Cancel and verify that the user was not deleted
    • Select the user for deletion again and choose Delete. Verify that the user was deleted.
  • #5988 - TLS 1.3 only

    • (HTTPS-only) On the Source Interface index page, verify via the Tor Browser Security Tab that the connection is encrypted not only as an Onion Service but also via TLS 1.3
    • (HTTPS-only) On the application server, inspect /etc/apache2/sites-enabled/source.conf and verify that it contains the line SSLProtocol all -SSLv3 -TLSv1 -TLSv1.1 -TLSv1.2
    • Install testssl.sh sudo apt install -y testssl.sh -t buster-backports in Tails, then run testssl <onion url>. Confirm that only TLSv1.3 is provided; all earlier protos are not.
  • #5979 - Remove expired key

    • verify that the old signing key with fingerprint 22245C81E3BAEB4138B36061310F561200F4AD77 is not present on the application or monitor servers, e.g. by running the command for s in app mon; do ssh $s sudo apt-key list; done from the Admin workstation
    • Verify that the new signing key 2359 E653 8C06 13E6 5295 5E6C 188E DD3B 7B22 E6A3 is presenton the servers
  • #6107 - Mute fwupd alerts

    • Verify that no OSSEC alerts related to fwupd were received
  • #5909 - Manually-transferred backup

    • Back up the instance using ./securedrop-admin backup
    • Copy the backup file to the application server using e.g. scp sd-backup.tar.gz app:/tmp/sd-backup-transfer.tar.gz
    • Verify that the command ./securedrop-admin restore --no-transfer sd-backup-transfer.tar.gz completes successfully, and that the local backup file is not transferred to the server during the playbook run.
  • #6110 - Repair Tails installer

    • (Tails version 4.18 or lesser):
      • Verify that ./securedrop-admin tailsconfig completes successfully and the Tails OS Updater starts without displaying errors
      • Verify that Tails can be successfully updated to the latest version.
    • (Tails latest version):
      • Verify that the command ./securedrop-admin tailsconfig completes successfully without triggering the Tails OS Updater.
      • on the admin workstation, edit /etc/os-release to change the TAILS_VERSION_ID to a pre-4.19 version
      • Run touch /usr/local/etc/ssl/certs/tails.boum.org-CA.pem
      • Run `./securedrop-admin tailsconfig and verify that it completes successfully
      • Verify that the contents of this file were added to /usr/local/etc/ssl/certs/tails.boum.org-CA.pem

Preflight testing

Basic testing

  • Install or upgrade occurs without error
  • Source interface is available and version string indicates it is 2.1.0
  • A message can be successfully submitted

Tails

  • The updater GUI appears on boot
  • The update successfully occurs to 2.1.0
  • After reboot, updater GUI no longer appears
Clone this wiki locally