Skip to content

Architecture

Jump to bottom
redshiftzero edited this page Nov 20, 2019 · 15 revisions

Overview

The SecureDrop Client is a GUI-based application for interacting with SecureDrop sources and their submissions on Qubes. It runs in a non-networked AppVM and launches other AppVMs as needed in order to view, process, manipulate, and export files from sources.

Inter-VM communication

It uses the securedrop-sdk, which is an API client written in Python for the SecureDrop journalist API. securedrop-sdk can be used to send HTTP requests/responses directly, or by proxying those requests through a networked AppVM.

sd-svs AppVM <- RPC (via qrexec) -> securedrop-proxy in sd-proxy AppVM

Code Structure

Entry point

  • app.py: configures and starts the SecureDrop Client application

Models

  • db.py: contains our database models

View

  • main.py: contains the QMainWindow class, which sets up the main application layout and connects other application widgets to the Controller
  • widgets.py: contains all application widgets except for QMainWindow

Controller

  • logic.py: contains the Controller class, which syncs the client with the server, creates and enqueues jobs on behalf of the user, and sends Qt signals to the GUI

Database and file storage

  • storage.py: contains functions that perform operations on the local database and filesystem

Queue

  • queue.py: contains the RunnableQueue class, a wrapper around Python's priority queue, and the ApiJobQueue class, a queue manager that manages two RunnableQueues on separate threads: one for downloading file submissions and one for everything else

Jobs

  • base.py: contains job interfaces that provide exception handling and a way to signal back to the controller whether or not a job was successful

Queue Architecture

A design goal of this application is for all API actions to occur via the queue (not true in the current codebase), of which we have two:

  • one for general network actions
  • one for downloading files

Each queue processes one job at a time. They are implemented via the RunnableQueue class. Each RunnableQueue contains a queue attribute which is simply Python's priority queue implementation (queue.PriorityQueue). This is used to prioritize more important jobs over others. One of the quirks of Python's Priority queue is that it does not preserve FIFO ordering of objects with equal priorities. A counter was added to our job objects to ensure that the sort order of objects with equal priorities is stable.

We have several jobs, in order of priority highest to lowest:

  • TokenInvalidationJob - unimplemented, ensures that API tokens are marked as invalid on the server (highest priority job)
  • PauseQueueJob - implemented, pauses the queue when authentication errors (due to the token expiring) or network timeouts occur
  • MetadataSyncJob - unimplemented
  • FileDownloadJob, MessageDownloadJob, ReplyDownloadJob - downloads files, messages, or replies (equal priorities)
  • DeletionJob - unimplemented, deletes sources
  • SendReplyJob - sends a reply to a source
  • UpdateStarJob - updates a source star (to starred or unstarred), which is used to indicate interest in a source
  • FlagJob - unimplemented, flags a source for reply (this ensures that they have a key generated for them next time they log in)

Synchronizing state with the SecureDrop server

There is consistent ordering of replies (from multiple journalists), messages, and files, set by the arrival of messages on the SecureDrop server, and clients synchronize to this. The ordering is performed on the client side using the file_counter of the conversation item instead of timestamps.

For pending or failed replies, they are stored persistently in the client database and are considered drafts. Draft replies contain:

  • a file_counter which points to the file_counter of the previously sent item. This enables us to interleave the drafts with the items from the source conversation fetched from the server, which do not have timestamps associated with them.
  • a timestamp which contains the timestamp the draft reply was saved locally: this is used to order drafts in the case where there are multiple drafts sent after a given reply (i.e. when file_counter is the same for multiple drafts).

Example of how replies and drafts work:

  1. Reply Q, has file_counter=2
  2. User adds DraftReply R, it has file_counter=2
  3. User adds DraftReply S, it has file_counter=2 and timestamp(S) > timestamp(R).
  4. DraftReply R is saved on the server with file_counter=4 (this can happen as other journalists can be sending replies), it is converted to Reply R locally.
  5. We now update file_counter on DraftReply S such that it appears after Reply R in the conversation view.

Notes and other docs

Clone this wiki locally