Skip to content

1.0.0 Test Plan

Jump to bottom
Kevin O'Gorman edited this page Sep 13, 2019 · 9 revisions

QA plan

  • NUC5s
  • NUC7s
  • Mac Minis
  • 1U servers in SF

1.0.0 QA Checklist

For both upgrades and fresh installs, here is a list of functionality that requires testing. You can use this for copy/pasting into your QA report. Feel free to edit this message to update the plan as appropriate.

If you have submitted a QA report already for a 1.0.0 release candidate with successful basic server testing and application acceptance testing sections, then you can skip these sections in subsequent reports, unless otherwise indicated by the Release Manager. This is to ensure that you focus your QA effort on the 1.0.0-specific changes as well as changes since the previous release candidate.

Environment

  • Install target:
  • Tails version:
  • Test Scenario:
  • SSH over Tor:
  • Onion service version:
  • Release candidate:
  • General notes:

Basic Server Testing

  • I can access both the source and journalist interfaces
  • I can SSH into both machines over Tor
  • AppArmor is loaded on app
    • 0 processes are running unconfined
  • AppArmor is loaded on mon
    • 0 processes are running unconfined
  • Both servers are running grsec kernels
  • iptables rules loaded
  • OSSEC emails begin to flow after install
  • OSSEC emails are decrypted to correct key and I am able to decrypt them
  • QA Matrix checks pass

Command Line User Generation

  • Can successfully add admin user and login

Administration

  • I have backed up and successfully restored the app server following the documentation here: https://docs.securedrop.org/en/latest/backup_and_restore.html
  • If doing upgrade testing, make a backup on 0.14.0 and restore this backup on 1.0.0
  • "Send Test OSSEC Alert" button in the journalist triggers an OSSEC alert and an email is sent.

Application Acceptance Testing

Source Interface

Landing page base cases
  • JS warning bar does not appear when using Security Slider high
  • JS warning bar does appear when using Security Slider Low
First submission base cases
  • On generate page, refreshing codename produces a new 7-word codename
  • On submit page, empty submissions produce flashed message
  • On submit page, short message submitted successfully
  • On submit page, file greater than 500 MB produces "The connection was reset" in Tor Browser quickly before the entire file is uploaded
  • On submit page, file less than 500 MB submitted successfully
Returning source base cases
  • Nonexistent codename cannot log in
  • Empty codename cannot log in
  • Legitimate codename can log in
  • Returning user can view journalist replies - need to log into journalist interface to test

Journalist Interface

Login base cases
  • Can log in with 2FA tokens
  • incorrect password cannot log in
  • invalid 2fa token cannot log in
  • 2fa immediate reuse cannot log in
Index base cases
  • Filter by codename works
  • Starring and unstarring works
  • Click select all selects all submissions
  • Selecting all and clicking "Download" works
Individual source page
  • You can submit a reply and a flashed message and new row appears
  • You cannot submit an empty reply
  • Clicking "Delete Source And Submissions" and the source and docs are deleted
  • You can click on a document and successfully decrypt using application private key

Basic Tails Testing

Updater GUI

After updating to this release candidate and running securedrop-admin tailsconfig

  • The Updater GUI appears on boot
  • Updating occurs without issue

1.0.0-specific changes

Note that it is not expected that a single tester test each one of the Tor onion services scenarios, please just indicate which scenarios you covered in the comment on the release ticket and the row at the end of the QA matrix (please fill the QA matrix in as you begin QA such that work is not duplicated).

From a 1.0.0 install:

Tor onion services: upgrade to v2

  • Do not rerun ./securedrop-admin sdconfig. Using the same site-specific as from before your upgrade to 1.0.0, run ./securedrop-admin install. V2 should still be enabled, and v3 should not be enabled.

Tor onion services: upgrade to v2+v3

Precondition:

  • Save the site-specific from v2 only. This will be used in a test towards the end of this section.
  • Perform a backup on v2.
  • rerun ./securedrop-admin sdconfig to enable v2 and v3 onion services, and then do ./securedrop-admin install. Then run securedrop-admin tailsconfig and check if the source and journalist desktop shortcuts have working v3 onion address.
  • Now disable SSH over Tor, rerun securedrop-admin install and ./securedrop-admin tailsconfig:
    • Verify that ~/.ssh/config contains IP addresses rather than Onion service addresses for the app and mon hosts
    • Verify that ssh app and ssh mon work as expected.
  • Use make self-signed-https-certs to generate self-signed certificates for testing, and run ./securedrop-admin sdconfig enabling HTTPS:
    • Verify that a warning is shown to the user indicating that they should update their certificate prior to sharing their v3 onion URL with users.
  • Use ./securedrop-admin restore to attempt to restore the v2 backup file created earlier.
    • Verify that the restore command fails with a message indicating that a v2 backup can't be used with an instance with v3 enabled.
  • Test multi-admin behavior. Conduct this test step after v3 is enabled on the server:
    • Back up site-specific, and copy the version from before the upgrade into place instead. Re-run ./securedrop-admin install, and verify that it fails with a user-friendly message, due to v3_onion_services=False.
    • Restore the v3 site-specific and move the tor_v3_keys.json file out of install_files/ansible-base. Re-run ./securedrop-admin install and verify that it fails with a message due to the missing keys file.
  • Restore the tor_v3_keys.json file. Perform a backup and restore against the v2+v3 install. The v3 onions should still be enabled after the restore.

Tor onion service: v3 only, no HTTPS

  • rerun ./securedrop-admin sdconfig to enable only v3 onion services, and then do ./securedrop-admin install. Now run ./securedrop-admin tailsconfig and check if the source and journalist desktop shortcuts has working v3 onion address. (#4708, #4677)
  • Now disable SSH over Tor, and re-run ./securedrop-admin install:
    • Verify that ~/.ssh/config contains IP addresses rather than Onion service addresses for the app and mon hosts
    • Verify that ssh app and ssh mon work as expected.
  • Run backup and restore to a new install. The v3 onions should be enabled.

Tor onion service: adding v3 interfaces with SSH over LAN

  • From a v2-only instance using SSH over LAN, upgrade to v3 only. You should continue to be able to SSH over LAN, and the v2 and v3 source and journalist interfaces should be available.

Deletion functionality

Testing detection and correction of disconnected submissions

Visit the source interface and send two messages. First we'll test a disconnected database record.

In your www-data shell:

cd /var/lib/securedrop/store
ls -laR

You should see the two message files. Remove one with rm.

cd /var/www/securedrop

  • ./manage.py check-disconnected-db-submissions should report There are submissions in the database with no corresponding files. Run "manage.py list-disconnected-db-submissions" for details.

  • ./manage.py list-disconnected-db-submissions should list the ID of the deleted submission, e.g. 2.

  • ./manage.py delete-disconnected-db-submissions should prompt you with Enter 'y' to delete all submissions missing files: -- reply y and you should see Removing submission IDs [2]... (the ID may differ).

Now we'll delete the remaining database record and verify that its disconnected file is detected. Still in your www-data shell:

sqlite3 /var/lib/securedrop/db.sqlite

Delete the submission record for the remaining message (substitute your filename):

delete from submissions where filename = '1-exhausted_overmantel-msg.gpg';
  • ./manage.py check-disconnected-fs-submissions should report There are files in the submission area with no corresponding records in the database. Run "manage.py list-disconnected-fs-submissions" for details..
  • ./manage.py list-disconnected-fs-submissions should show a list like: /var/lib/securedrop/store/B3A5GPU4OHPQK736R76HKJUP5VONIOMKZLXK77GPTGNW7EJ63AY5YBX27P3DB2X4DZBXPX3LGBBXAJZYG3HQRHE4B6UE5YYBPGDYZOA=/1-exhausted_overmantel-msg.gpg
  • ./manage.py delete-disconnected-fs-submissions should prompt you to delete that file. Do so and it should be deleted.

Testing automatic requeuing of interrupted deletions

Establish two SSH connections to the app server. In one, become root with sudo su - and in the other become www-data with sudo -u www-data bash. In the www-data shell:

Activate the securedrop-app-code virtualenv: . /opt/venvs/securedrop-app-code/bin/activate

cd /var/www/securedrop

Create a big file that will take a while to delete: dd if=/dev/zero of=/var/lib/securedrop/store/bigfile bs=1M count=1000

Submit a job to delete it:

python3
>>> import rm
>>> import worker
>>> q = worker.create_queue()
>>> q.enqueue(rm.secure_delete, "/var/lib/securedrop/store/bigfile")

Exit Python.

In the root shell: Reboot, then reconnect.

Look at the rqrequeue log: less /var/log/securedrop_worker/rqrequeue.err -- at the end you should see lines like this:

2019-08-08 17:31:01,118 INFO Running every 60 seconds.
2019-08-08 17:31:01,141 INFO Requeuing job <Job 1082e71f-7581-448c-b84b-027e55b4ef8e: rm.secure_delete('/var/lib/securedrop/store/bigfile')>
2019-08-08 17:32:01,192 INFO Skipping job 1082e71f-7581-448c-b84b-027e55b4ef8e, which is already being run by worker rq:worker:6a6b548310f948e291fa954743b8094f

That indicates the interrupted job was found and restarted, but was left alone at the next check because it was already running. The job should run to completion, /var/lib/securedrop/store/bigfile should be deleted, and the rqrequeue log should start saying:

2019-08-08 17:33:01,253 INFO No interrupted jobs found in started job registry.
  • Verified the requeue behavior

Testing OSSEC reporting of disconnects

Create a file under /var/lib/securedrop/store with touch /var/lib/securedrop/store/testfile. If you don't feel like waiting a day for the OSSEC report, you can edit /var/ossec/etc/ossec.conf, look for check-disconnect, and reduce the <frequency>, then service ossec restart.

  • An OSSEC alert was sent indicating a disconnect had occurred.
  • I otherwise got no manage.py notifications about this functionality.

Design changes:

  • The design changes from https://github.com/freedomofpress/securedrop/pull/4634 are present on the Source Interface
    • The default logo is updated
    • Text colours are updated.
  • Desktop shortcut icons are updated in the Admin Workstation.
  • The updater logo and icon has been updated in the Admin Workstation

Miscellaneous other changes

  • On the Application Server, inspect the version of Python by running ps -aux | grep python

    • /opt/venvs/securedrop-app-code/bin/python is used by the application instead of /usr/bin/python.
    • /opt/venvs/securedrop-app-code/bin/python version is Python 3.5
    • Other running python processes use Python 3 with the exception of supervisord

  • Journalist notifications continue to work as expected on 1.0.0.

  • Both app and mon servers are running Tor version 0.4.x (#4658).

  • Login as a journalist, and via another admin account reset the password of the journalist, this should invalidate the journalist session, and the journalist must relogin (#4679)

  • There are no linux-image generic packages installed (non-grsec kernels) (apt list --installed | grep linux-image) (#4641)

  • Shortly after uploading a file via the Source Interface, a checksum is added to the submissions table (managed via rq)

Preflight

  • Ensure the builder image is up-to-date on release day

These tests should be performed the day of release prior to live debian packages on apt.freedom.press

Basic testing

  • Install or upgrade occurs without error
  • Source interface is available and version string indicates it is 1.0.0
  • A message can be successfully submitted

Tails

  • The updater GUI appears on boot
  • The update successfully occurs to 1.0.0
  • After reboot, updater GUI no longer appears
Clone this wiki locally