Skip to content

Testing Scenarios

Jump to bottom
Olivier Goffart edited this page Jun 7, 2017 · 50 revisions

Introduction

This page describes general testing required basically for each client release.

For individual client releases, there are more specific test plans:

Add manual test here to this page that are

  • critical for the applications basic functionality
  • features that might easily regress, esp. bugs that you have fixed (add bug number)
  • not specific to one release

Use this as a guide to test releases in the beta phase. All tests should be described in a way such that they can be performed by anyone (no hard developer knowledge should be required). It is strongly recommended never to test with production data. always use at least a test-user. These tests are designed to make the Client explode or at least push it to its limits. You have been warned!

Note: Syncing should be tested at least against the newest and oldest supported owncloud server.

Where to find directories on the various OSes

Windows Mac OS X Linux
User app settings %LOCALAPPDATA%\ownCloud ~/Library/Application Support/ownCloud ~/.local/share/data/ownCloud
System-wide app directory %ProgramFiles(x86)%\ownCloud /Applications/ownCloud.app/Contents/Resources /usr/bin (binary)
/etc/owncloud (settings)
Default sync directory %USERPROFILE%\ownCloud ~/ownCloud ~/ownCloud

Test classes

Installer/Uninstaller

  • Install ownCloud
  • expected: files to be found in System-wide app directory.

Setup

  • Check if hostnames are accepted correctly by the input field
  • Check if self-signed certificates are being accepted

Syncing

Local Sync Folder permissions

  • When creating a local folder either by following the "new account" or the "new folder sync connection" wizard, the permissions on the folder should be 700: rwx------ to avoid different users seeing the contents of those.
  • When using an existing folder, regardless of the resolution strategy picked in the wizard or the previous permissions of the folder, resulting permissions should also match 700.

User stop syncing

(By pausing or logging out) Expected: Client stops syncing Error: Client crashes, Client stays in sync state (check with --logwindow

  • Stop syncing while doing local discovery
  • Stop syncing while doing remote discovery
  • stop syncing while uploading a file
  • stop syncing while downloading a file
  • stop syncing while deleting files
  • stop syncing while renaming files

Test resuming of uploads

  • Add a big files locally (100Mb or more) and look at the server logs: it should start sending chunks
  • Kill the client while syncing.
  • Restart the client: it should resume from the last sent chunk, and stop when all the chunk are transfered (observe that on the server log)
  • Do the same but instead of killing the client, shut the server off or turn off the network

Test resuming download

  • Add a big file on the server
  • Look at the progress bar of the sync and kill the client when it is almost over
  • Restart the client
    • The sync should resume from what it already downloaded.
    • Known bug: the progress still show it has too download it full
    • But after the remaining megabytes have been downloaded, the progress bar should jump to the end.

Modifying files when the sync is running

  • Add a new directories full of files and let it sync.
  • While syncing, rename directory on the client.
  • After two sync, the directory fust have been renamed on the server and must not re-appear on the client, and all the files should be there.
  • Now do the same, but delete the directory instead of renaming
  • Do the same, but rename the directory on the server when syncing
  • The same but delete the directory on the server

Test shared folder

  • Have a read only Shared/ folder
  • Adding a file there should cause an error. The error should be blacklisted after a few sync.
  • Editing a file should cause an error saying that a conflict file has been created and the original file restored. Verify it worked
  • Removing a file should restore the file
  • Renaming a file should restore the original, and show an error for the new one.
  • Rename Shared/ to something else : Syncing should move Shared/ back
  • Move a subfolder of Shared outside of Shared : The original files should be restores at their original location, and the new directory uploaded to the server
  • Remove a subfolder : The original files hsould be restored

Network-related

  • Network goes away
  • Expected: Client recovers
  • Error: Client crashes, client stays in error state
  • Network goes online again:
  • Expected: the sync resumes

Crash Recovery

  • client crashes (kill -9, force end via task manager)
  • Expected: client recovers
  • Error: client shows error message, DB corrupts, settings are forgotten
  • Important: check behavior also with clean settings directory. Does the client forget its credentials?

Server interaction checks

  • Client must not initiate new session per request, make sure cookie gets passed along.
  • Remove DB.
  • Expected results: no changes after initial sync.
  • Tolerated: some conflicts when server and client are not time-synced
  • Error: Conflict files get uploaded to server

Upgrade (where applicable)

Upgrade from Previous Client Version

For this test, the server stays the same.

On Windows: Install the previous version of the client. Configure some acconts and folder and other settings. Download the new installer and start it while the previous is running. It will ask if it should remove the older installation, test with both choices. Reboot after every update and see if the system start works fine. Check that the settings and folder configuration are kept from one version to the next.

Upgrade from previous server version

For this, the client version is the one that should be tested and stays the same. Install on older server version, sync. After that, update the server and let it perform its update business. The client needs to handle that gracefully.

Credential Management

The following test cases should be checked.

First startup

This is the situation new users have: There is no former installation with credentials stored somewhere.

  • Check if the account has set credentials in the password storage which is Registry on Windows, Wallet system on Linux and the System Password storage on MacOS. Remove all ownCloud related entries there. Do that after ever test here.
  • Test if the user can connect to a http server with correct credentials.
  • Test if the user can connect to a https server with correct credentials.
  • Test what happens if user connects with wrong user.
  • Test what happens if user connects with wrong password.
  • Test what happens if user presses "Cancel" without having connected.
  • Verify that user can not press "Finish" before a successful connection was established.
  • Verify that if we bo back and enter different URL everything works correctly

Update

Test authentication after the user updated from a previous version. Basically repeat the tests from before but with existing credentials in the credential store.

  • Check if the client starts without asking for password on first start after update.

Password change, without URL change.

Test the case that the user wants to change her password. For that, run the configuration dialog and just change the users password. For that, first change the password on the server using the web interface.

  • After the password change on the webinterface, check the client's reaction when it tries to connect with the old, now wrong credentials. Start the client with the wrong credentials.
  • Change the credentials on the server while the client is running. Check the reaction.
  • Test if the new password is accepted and appears in the credential store.
  • Test if the client connects to the server and starts syncing after user pressed "Finish".
  • Test if the old configuration remains unchanged after user pressed "Cancel".
  • Test if the proxy settings are the same as before after reconfiguration.
  • Go back in the configuration wizard and check if the password is still consistently used.
  • Test with a wrong password.

Password change, with URL change

Repeat the tests above, but also with the url changing.

Wrong password

  • Sing out. (from the menu in the systemtray)
  • Sign in: It should ask for the password
  • Press cancel: it should stay disconnected and not ask for the password
  • Sign in, and enter the wrong password: it should ask for the password again
  • Enter the wrong password a couple of times: it must always ask for the password
  • Enter the right password: It should do the sync and connect and not ask for the password anymore.

Shibboleth

  • Change the account to use an account on the shibboleth test server
  • Connect and start a sync.
  • Try sign in / sign out
  • Try wrong password, try closing the shibboleth window without loggin in.
  • Try loggin with a different user. This should not be allowed

OAuth2 (client 2.4)

  • Add a new account from the wizard with an OAuth2 enabled server.

    • One should not be able to go "back" once the login is done

  • Log in / Log out situation

  • Test possible error conditions:

    • The user does not enter login for a timeout of 5 minutes: The wizard (or the setting dialog) should show an error and it should be possible to retry
    • After the login is successful, if there is an error getting the access token (30 second timeout, or the server reject or client id/ secret password or if the code is invalid), the wizard or setting dialog should show an error and it should be possible to retry. This can also be tested by going to the redirect url in another tab. [http://localhost:xxxx/?code=invalid]
    • (Connecting to the redirect_uri with with browser without a code in the URL should not have any effect)

  • When the access_token times out (usually after one hour), it should be automatically refreshed using the refresh_token. All should go smooth even if it happens in the middle of a sync.

  • Restart the client: The refresh_token should have been stored on the key chain and the connection should be set up automatically. (Unless the user logged out manually).

  • Test that if the OAuth2 app is enabled or disabled on the server while an account is already configured, the client should automatically fallback to the right authentication method for the next connection.

Multiples folders

  • Add multiples folder : Check that they sync properly
  • Remove a folder : There should not be crashes.

Activity log

  • Look at the activity log.
  • Click or double click on items. It should open the explorer with the file selected.

Quota

  • Check that the quota displayed on every account page is correct compared to what the server says it is.
  • Change the quota on the server. Make sure that the quota is updated within 30 seconds on the quota indicator on the client. This need to be tested if the ui is visible when the change happen, or hidden when the change happen.
  • After a sync that uploaded files, the quota should be updated within 30 seconds.

Windows Office Files

  • Save an office file under windows into the sync dir. Keep the file open in the office app. Every time the file is saved in the office app, the file should be synced.

Big File modified during Upsync

  • Put a big file into the sync dir. While it is synced up, change it. The client should realize the change, stop syncing and redo it later.

Command Line Client

  • Check that no username/password is logged in cleartext during operation.

☁️

Clone this wiki locally