testing.md

What is testing all about?

• The goal of testing:
  • To test that users can actually use an app.
  • To test that an app doesn't break when:
    • Bad data are entered.
    • Unexpected actions are performed.
• Anticipate what would happen when a user:
  • Makes a typo.
  • Tries to save an incomplete form.
  • Uses the wrong API.
• Check if someone can easily compromise data or gain access to a resource they're not supposed to.
• A good testing suite should try to break your app and help understand its limit.

Tests -- categorized according to their execution type

• Automated tests: performed by a machine that executes a test script that was written in advance.
• Manual tests.

A quick run down for the ones I felt are more common.

	Unit testing	Integration testing	E2E testing	Smoke testing
Cost	Cheap	Expensive	Expensive	Moderate
Complexity (setup & maintain)	Easy	Hard	Harder	Moderate
Objective	Individual methods/functions work well	Different modules/services work well together	Verifies that various user flows work as expected	The major features of the system are working as expected.

Automated tests

• Unit testing.
  Breaking down unit tests.

  • Cheap.
  • Very low level.
  • Testing individual methods/functions.
  • A unit is the smallest testable component of an application.
• Integration Tests.
  Breaking down integration tests.

  • Expensive.
  • Different modules/services used by the app work well together.
• Functional tests.
  Breaking down functional tests.

  • Black-box testing
  • Focuses on the business requirements.
  • Checks output of an action, not the intermediate states.
• E2E Testing.
  Breaking down E2E tests.

  • Expensive.
  • Hard to maintain.
  • Replicates a user behavior.
  • Various user flows work as expected.
  • A few key E2E tests. Rely more on lower level types of testing (unit and integration tests).
• Acceptance Testing.
  Breaking down acceptance tests.

  • Replicates a user behavior.
  • Measure the performance of the system.
  • Rejects changes if certain goals are not met.
  • Verify if a system satisfies business requirements.
• Performance testing.
  Breaking down performance tests.

  • Evaluates how a system performs under a particular workload.
  • Measures the reliability, speed, scalability, and responsiveness.
• Smoke Tests.
  Breaking down smoke tests.

  • Basic tests.
  • Quick to execute
  • Checks the basic functionality of the app.
  • Guarantee that the major features of the system are working as expected.
  • Useful right after:
    • A new build is made & we wanna decide whether or not you can run more expensive tests.
    • A deployment to make sure that the application is running properly in the newly deployed environment.
• Component Testing.
  Breaking down component tests.

  • Breaking applications into smaller logical units called components.
  • Range from fairly small (like a button) to more complex (like a registration form).
  • Tend to be easily testable.
  • Written by the developers working on the component.
  • Component tests do nothing to ensure that all the layers of your app are working well together.
  • Common scenarios for component tests:
    • Testing a date picker works properly.
    • A form shows/hides specific sections based on input.
  • For this you can use Storybook or Cypress.
• Regression Tests.
  Breaking down regression tests.

  • Re-running functional and non-functional tests to ensure that previously developed and tested software still performs as expected after a change.
  • If not, that would be called a regression.
  • A software regression is a type of software bug where a feature that has worked before stops working.
  • Things like:
    • Bug fixes.
    • Software enhancements.
    • Configuration changes
  • Some tools used for regression tests:
    • Selenium.
    • Ranorex Studio.

  

Tip

Functional VS integration.

• An integration test may simply verify that you can query the database.
• While a functional test would expect to get a specific value from the database as defined by the product requirements.

Manual tests

• Exploratory testing:
  • Objective: uncover non-obvious errors.
  • An exploratory testing session should not exceed two hours.
  • With a clear scope -- helping testers to focus on a specific area of the software.

Tests -- categorized according to their scope

• Black-box:
  • The tester is not concerned with the implementation details.
  • Focused on validating the functionality based on the provided specifications/requirements.
    • What's input and output.
• White-box:
  • Focuses on code coverage:
    • Path coverage: 3 / 0 throws an error for a division operation.
    • Statement coverage: 3 + 3 equals 6 for an additional operation.
  • Analyze the
    • Internal structures the used data structures.
    • Internal design.
    • Code structure.
  • AKA:
    • Open box testing.
    • Glass box testing.
    • Clear box testing.
    • Structural testing.
    • Transparent testing.

Most important aspects of the two, put together side by side.

	Black Box Testing	White Box Testing
Objective	Testing the functionality of the software.	Ensuring the internal code is correct and efficient.
Scope	Overall behavior.	Internal logic.
Performed by	QA	Developer
Development methodology	BDD (Behavior-Driven Development)	TDD (Test-Driven Development)
Granularity	Just test CBP and the positive scenarios.	Go for code coverage -- Do not go overboard, sometimes it does not worth having 100% code coverage.

E2E Testing

• A technique.

Aspects	Considerations
Ensures the app is functioning as a cohesive whole.	More difficult to set up, run, and maintain
Tests are written in code with an API that simulates the steps that a real user would take.	Provision testing infrastructure in CI
Can be written by developers/QA Teams.	Testing certain scenarios require more setup
Tests the app from the web browser through to the backend, as well as testing integrations with 3rd-party APIs and services.

Scenarios for E2E

• Validating critical business path (CBP), things such as authentication and purchasing.
• Ensuring data is persisted and displayed through multiple screens.
  • This means verifying that the saved data appears correctly across different parts of the application. For example, if a user enters information on one page, that same information should be accurately displayed when the user navigates to another related page.
• Running Smoke Tests and System Checks before deployment.

Refs

The different types of software testing

Also published:

• At dev.to

Learn more

• Understanding test strategy in software testing.