design-docs: describe process and add examples (open-telemetry#114)

design-docs/00000-example-docs/README.md

@@ -0,0 +1,3 @@
+This directory contains two old design documents from times before the agent
+was donated to OTel and open-sourced in the process. The documents are provided
+primarily as examples of how a design document can be structured.

design-docs/00000-templates/long-form-design-doc-template.md

@@ -0,0 +1,163 @@
+Long-Form Design Doc Template
+=============================
+
+> [!NOTE]
+>
+> This template is intended for use when a long-form design document is
+> necessary. A document using this template should provide sufficient high level
+> context to understand the problem as well as detailed proposals with supporting
+> data.
+
+# Meta
+
+- **Author(s)**: Name of the author(s)
+- **Start Date**: Start date of the design doc
+- **Goal End Date**: Intended end date for the design process
+- **Primary Reviewers**: A list of at least two primary reviewers
+
+# Abstract
+
+A reasonably concise abstract so that a reader can get an idea of what
+the document is about.
+
+# Introduction
+
+A good introduction into the problem that we are trying to solve. This
+introduction should not assume much background knowledge from the
+reader, but also not re-hash everything from ground up; when any
+external documentation can be referenced, this should be given
+preference over re-hashing things. The introduction should have the
+following sections
+
+## Context
+
+Present ideas, dependencies, systems and general context referred to in
+the doc.
+
+### Constraints
+
+Document any relevant constraints that any/all solutions must adhere to.
+Try to be as explicit as possible here, as constraints that are obvious
+to you may be less so to another reader.
+
+As an example, when working on a task that relates to the eBPF tracers
+there are a number of constraints that come into play depending on the
+kernel version one wants to support. In this case it would be
+appropriate to list each of these constraints, and the kernel versions
+they are relevant to.
+
+### Related (Sub-)Problems
+
+If there is a set of sub-problems that need to be explained, and
+possibly solved, as part of one or more of your solutions it can make
+sense to address them up-front and independently. Include solutions to
+such problems here as well, if appropriate.
+
+## Problem Statement
+
+What the problem is, why the problem exists, why do we need to solve it.
+
+**For particularly significant tasks where it is important to solicit
+feedback as early as possible it often makes sense to write this section
+and the success criteria first. This allows for feedback to be solicited
+before writing the rest of the document, in order to ensure everyone is
+on the same page before major time investment.**
+
+## Success Criteria
+
+By far the main pitfall in many design docs is not clearly defining what
+"success" means. This sometimes can lead the author to meander into
+less-important areas, or ignore an important aspect completely. This can
+also lead the reviewers to be on a completely different line of thought,
+orthogonal to the doc.
+
+This section is more or less what defines the outline and then sets the
+stage for the sections/subsections of the document. It's a reality
+check that allows the authors and the reviewers to agree on the basis
+for the design, and how it should be reviewed. It should help to make
+sure everybody is focused on the same thing.
+
+Usually this should be presented as a concise series of bullet points.
+
+## Scope
+
+This expands on the "success criteria", but helps to clear up possible
+confusion about what the design doc is about and is not about. It's
+usually extremely short, a few bullet points. The most useful info here
+is usually "XYZ is not in scope" to avoid ambiguity. This ensures we
+don't make incorrect assumptions when reading the doc
+
+# Proposed Solution(s)
+
+Ideally, there should be more than one proposed solution. The document
+should list the various solutions, and then go into some depth about
+their drawbacks and advantages.
+
+Something to watch out for here is a situation where a problem has N
+sub-problems, each with their own M alternative solutions. In that case
+it is usually unnecessary, and unreadable, to enumerate all possible
+$N \cdot M$ permutations. Assuming it is feasible, outline the possible
+solutions to each sub-problem separately, and present a very limited
+number of combined solutions, if any.
+
+## Author's Preferred Solution
+
+After presenting the merits of each solution, you should ideally give a
+hint as to what your preferred option is, and why. This achieves 2
+goals:
+
+-   It ensures solutions are evaluated against each other, despite
+    having pros and cons that are not easily comparable. Discussing this
+    can provide helpful context to the reader.
+-   Writing a justification is a good reality-check, ensuring the design
+    doc achieves its goal and provides enough information to make a
+    decision.
+
+
+In case you do not have a preferred solution: to help the reader, you
+can describe an appropriate thought process for comparing solutions:
+
+-   Which pros/cons are more important than others?
+-   What features or constraints are more important?
+
+We have a finite time/energy budget for assessing designs and providing
+this information will help you and the reviewers to prioritize.
+
+# Testing Strategy
+
+Document here how the solution itself will be tested, as well as how the
+solution may impact the testability of other components that make use of
+it.
+
+The testability of a component itself, as well as its impact on the
+testability of other components that make use of it, are important
+factors to weigh when choosing between alternate solutions. If it is the
+case that your proposed solutions differ in terms of how testable they
+are then it may make sense to lift this section into the proposed
+solutions, and repeat it for each.
+
+## Testing of Proposed Solution Itself
+
+Outline how the proposed solution will be unit tested and integration
+tested. For many implementations this is straightforward and obvious,
+but not always. If there are likely to be any difficulties in testing
+the solution then outline them here, as well as solutions, if any.
+
+## Impact on Testing of Other Systems/Components
+
+Discuss whether or not the proposed solution will impact the testability
+of systems that make use of it, and how. For example, if the solution
+involves producing a component that would be difficult to mock out for
+the purposes of testing something that makes use of it, then explain
+that here.
+
+# Plan to Acquire Missing Data (Optional)
+
+Plan to acquire missing data. Often, some data is missing to properly
+evaluate the advantages and disadvantages, and this section details what
+data needs to be gotten and how.
+
+# Decision
+
+When all the data is there, a "Decision" section which details what
+solution was decided on.

design-docs/00000-templates/short-form-design-doc-template.md

@@ -0,0 +1,50 @@
+Short-Form Design Doc Template
+==============================
+
+> [!NOTE]
+>
+> This template is intended to propose problems and solutions for which a
+> fully fledged design document is likely not necessary. A document made from
+> this template will have less high level context and detail than a long-form
+> document.
+
+# Meta
+
+- **Author(s)**: Name of the author(s)
+- **Start Date**: Start date of the design doc
+- **Goal End Date**: Intended end date for the design process
+- **Primary Reviewers**: A list of at least two primary reviewers
+
+# Problem
+
+Clearly describe the problem.
+
+# Success Criteria
+
+By far the main pitfall in many design docs is not clearly defining what
+"success" means. This sometimes can lead the author to meander into
+less-important areas, or ignore an important aspect completely. This can
+also lead the reviewers to be on a completely different line of thought,
+orthogonal to the doc.
+
+This section is more or less what defines the outline and then sets the
+stage for the sections/subsections of the document. It's a reality
+check that allows the authors and the reviewers to agree on the basis
+for the design, and how it should be reviewed. It should help to make
+sure everybody is focused on the same thing.
+
+Usually this should be presented as a concise series of bullet points.
+
+## Scope
+
+This expands on the "success criteria", but helps to clear up possible
+confusion about what the design doc is about and is not about. It's
+usually extremely short, a few bullet points. The most useful info here
+is usually "XYZ is not in scope" to avoid ambiguity. This ensures we
+don't make incorrect assumptions when reading the doc
+
+# Proposed Solution
+
+A proposed solution for the problem. Even though this is a short-form
+design document you must still provide enough information to allow the
+reviewers to understand and assess what you are proposing.

design-docs/README.md

@@ -0,0 +1,58 @@
+Profiling agent design documents
+================================
+
+This directory contains design documents that discuss major additions of changes
+to the profiling agent's architecture.
+
+### When to write a design document?
+
+Invoking the design document process can be beneficial if
+
+- you are proposing changes that have significant impact on multiple
+  sub-components of the profiling agent.
+- you are proposing a major addition and want to make sure that it will
+  actually be accepted upstream the way that you plan to implement it without
+  writing the code first (and thus risking need for major reworks later).
+- your proposed changes require a significant amount of context to understand
+  for reviewers. A short design document can help to clarify the current state
+  and the state that you'd like to move towards in these cases.
+- you'd like to incrementally apply reworks over the course of multiple
+  PRs, to provide extra context for reviewers to understand what the end
+  goal is supposed to look like. In simpler cases that can also be achieved
+  with a tracking issue.
+
+The above are guidelines: there is no hard rule on when a design document is
+necessary. When in doubt, please feel free to create an issue and quickly outline
+what you want to do in order to clarify whether a document is needed or not!
+
+### Creating a New Document
+
+- Create a new git branch forking from latest `main`
+- Create a directory in the `design-docs` folder
+  - The directory name should follow a format like `00000-my-doc-title`
+  - The intention for creating a directory per document is to allow bundling
+    media (images, diagrams, drawings) with the document
+  - The 5 digit number is included to make it easy to spot recent documents and
+    to order the documents by their recentness when viewed in the GitHub UI
+  - When initially writing the document, the ID is set to `00000`
+- Copy one of the templates for design documents from the (`00000-templates`)[./00000-templates]
+  directory into into the newly created directory, naming it `README.md`
+- Write the design document by following the instructions in the template
+- Once reaching draft quality, create a draft PR
+  - Add the `design-document` label
+  - Include a link to the rendered document. GitHub hides away the ability to
+    properly view rendered documents behind multiple clicks, so adding a direct
+    link spares others some work. After creating the PR, this link can be
+    obtained by clicking "Files changed", searching the `README.md` file,
+    clicking the three dots on the right, selecting "View file" and then
+    switching the current view to track your branch via the drop-down to the
+    left of the file path. The last step ensures that the link automatically
+    tracks updates when you push fixup commits later.
+- Rename the directory according to the PR ID that was allocated, e.g. if the
+  allocated PR ID is 1234 then `00000-my-doc-title` becomes
+  `01234-my-doc-title`
+- Once you're happy with the state of the document:
+    - Mark the PR as ready for review and tag at least 2 maintainers for review
+    - Additional people who should be aware of the document but whose review
+      isn't mandatory can be notified by mentioning their username in the PR's
+      description as `CC @username`