From 6574ee5c70460f67979fac1106245a2bbf0073c8 Mon Sep 17 00:00:00 2001 From: Daniel Kamkha Date: Mon, 31 May 2021 15:56:35 +0200 Subject: [PATCH] Add more Gluecodium internal documentation (#918) * Add more Gluecodium internal documentation Added Gluecodium internal documentation about the testing approach and about the release process. Resolves: #903 Signed-off-by: Daniel Kamkha * Update testing.md --- docs/internal/release_process.md | 45 +++++++++++++++++++++++++++++ docs/internal/testing.md | 49 ++++++++++++++++++++++++++++++++ 2 files changed, 94 insertions(+) create mode 100644 docs/internal/release_process.md create mode 100644 docs/internal/testing.md diff --git a/docs/internal/release_process.md b/docs/internal/release_process.md new file mode 100644 index 0000000000..29e8bf061d --- /dev/null +++ b/docs/internal/release_process.md @@ -0,0 +1,45 @@ +Gluecodium release process +========================== + +This document describes the approach to versioning and release process of the Gluecodium process. + +Semantic versioning +------------------- + +Gluecodium loosely follows the approach to version numbers known as "semantic versioning". Quoting +[semver.org](https://semver.org): + +> Given a version number MAJOR.MINOR.PATCH, increment the: +> 1. MAJOR version when you make incompatible API changes, +> 2. MINOR version when you add functionality in a backwards compatible manner, and +> 3. PATCH version when you make backwards compatible bug fixes. + +Gluecodium versions mostly follow this logic. The exceptions are based on the size of the feature being added: +* for *very small* features only PATCH version is incremented, even if it's not a bug fix. +* for *very big* features (e.g. a new generator) the MAJOR version is incremented, even if it's not breaking. + +Changelog +--------- + +Committed user-visible changes to Gluecodium are (manually) tracked in the [CHANGELOG.md](../../CHANGELOG.md) file. Each +new change should be briefly described at the top of the file, under `Unreleased` section. Most changes fall either into +`Bug fixes` subsection or into `Features` subsection. There are also rarer subsections for breaking changes, for +deprecations, and for removals. + +Committed changes that are not user-visible (e.g. refactoring) are not tracked in the changelog. + +Release process +--------------- + +Gluecodium has no fixed release schedule. Most MINOR and PATCH releases happen on-demand, according to requirements from +internal (in-company) users of Gluecodium. For bigger features (all MAJOR releases, and some MINOR) the release happens +as soon as the feature is ready. + +Each release raises the current Gluecodium version in a dedicated commit. The same commit should have the current +`Unreleased` section of the changelog converted into the `Release MAJOR.MINOR.PATCH` (also have a "release date" line +added there). + +After the version change is committed, a release tag needs to be created manually on the +[Releases](https://github.com/heremaps/gluecodium/releases) page. The convention is to copy-paste the current release +section from the changelog into the "description" field: this way the users of Gluecodium, who subscribed to release +notifications, also get the release notes for this release. diff --git a/docs/internal/testing.md b/docs/internal/testing.md new file mode 100644 index 0000000000..080664db52 --- /dev/null +++ b/docs/internal/testing.md @@ -0,0 +1,49 @@ +Gluecodium testing approach +=========================== + +This document describes the types of automated tests in Gluecodium project. + +Unit tests +---------- + +Unit tests in Gluecodium are small-scale white-box tests, mostly covering the supporting infrastructure, not the main +logic in generators. Most logic in generators is either expressed in Mustache templates (which cannot be unit-tested) or +is based on LIME model tree traversal (which requires rather complex setup of mocks/fakes for fine-grained testing). +Instead, generator logic is tested with large-scale white-box tests (see `"Smoke" tests` below). + +Currently, the following groups of unit tests exist: +* a few tests for utility classes in :lime-runtime Gradle project +* a few regression tests for specific use cases in :lime-loader Gradle project +* tests for template helpers in :gluecodium Gradle project +* tests for topological sorting algorithm in :gluecodium +* tests for various validators in :gluecodium + +"Smoke" tests +------------- + +"Smoke" tests in Gluecodium are large-scale white-box tests where the output code is generated based on given IDL files +and then this output is compared to existing reference files. It would be more correct to call these "acceptance" tests, +as "smoke" tests are usually just a few in number - which is not the case for Gluecodium "smoke" tests. + +The "smoke" tests are implemented as a single parameterized unit test `SmokeTest.kt`. The test auto-loads its test data +from the "resources/smoke" sub-directory. It sorts out the files into "input" IDL files and per-generator "reference" +files based on directory structure: "/input" for IDL files (and sometimes additional files like options or +name rules), "/output/" for reference files. If the "output" directory of some test case does +not have a sub-directory for a particular generator, the test case is ignored for this generator, but there is no +failure. + +Changes affecting generated code require updating existing "reference" files, even if no new "smoke" tests are created. +This ensures "smoke" tests are passing, and also demonstrates the changes to "real" code to the reviewers of the change. + +Functional tests +---------------- + +Functional tests in Gluecodium ("functional-tests" directory) are large-scale black-box tests where the output code is +generated based on given IDL files, compiled with the tools appropriate for each corresponding language, and then run +through a set of unit tests which use the compiled code as a library. While the testing happens through unit tests, with +regard to Gluecodium itself these tests are black-box tests, since they are using Gluecodium as a tool, like a user +would. + +These tests could be run locally (see [Development tools](development_tools.md) for the list of required software). They +are also run on CI (currently GitHub Actions CI, see the [config](../../.github/workflows/functional-tests.yml) for +details).