Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update builder specification for v0.11.0 #6

Closed
181 changes: 152 additions & 29 deletions builder.md
Original file line number Diff line number Diff line change
@@ -1,57 +1,180 @@
# Builder Guidelines
**Version 0.10.0**
# Base16 builder specification
**Version 0.11.0**

A base16 builder is an application that can build syntax highlighting definition files for text editors by using base16 scheme files which contain a collection of colours and base16 template files which contain syntax highlighting rules.
*The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).*

Builders are designed for theme maintainers' ease of use. Theme maintainers should provide built versions of their theme so the end user doesn't need to be aware of the builder.
## 1. Introduction

## File Structure
"Building" refers to replacing placeholders on base16 _[template
files](./file.md)_ with colors from base16 _[schemes](./styling.md)_.

### Schemes Repository
A template is a mustache file that acts as a "blueprint": it represents how to
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
output the scheme into other file formats, or (more commonly) how to output a
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
configuration file to theme a specific software. For example: a [vim
template](https://github.com/base16-project/base16-vim/blob/main/templates/default.mustache).

The schemes repo should either be stored in a common location (perhaps referred to by environment variable or command line flag) or dynamically embedded in the builder.
A scheme is a yaml file that represents a palette of 16 colors: to be replaced
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
on the placeholders the template files hold. For example: the [solarized
scheme](https://github.com/base16-project/base16-schemes/blob/main/solarized-dark.yaml)

- `/*.yaml`
A base16 builder is, essentially, an application that somehow implements the
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
_building_ feature specification: filling the placeholders on templates with
scheme colors.

### Template Repository
Builders are categorized into:
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved

Each template repository should have a templates folder containing a config.yaml and any needed mustache template files.
- _reference tooling_: These follow the spec closely and are
designed for unopinionated lower-level ("plumbing") usage, specifically for
scripting and as component to build more complex base16 applications. High
quality reference builders will usually be adopted by the base16-project
organization;
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
- _extended tooling_: These are built for end users and are usually tailored
for a usecase. This includes GUI apps, ergonomical CLI tools, scheme
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
managers, web apps, and more. These apps are encouraged to use (or create)
reference implementations as libraries, but may also implement the builder
feature themselves. All of the following `MUST`s can be replaced by `SHOULD`s
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
when writing this kind of tooling.

- `/templates/*.mustache` - A template file (there may be multiples of these)
- `/templates/config.yaml` - A template configuration file
## 2. Template variables spec

## Workflow
The first thing a builder needs to do is parse all the scheme files from the schemes repository (as defined in the [file guidelines](https://github.com/chriskempson/base16/blob/master/file.md)). All files matching the pattern `*.yaml` should be loaded from the root of the schemes repository.

When building a target template repository, a base16 builder should first clear out any old output. Next, for all templates defined in the template repo's config file (located at `/templates/config.yaml`), the builder should iterate through all the defined schemes and output matching files. The built filename should look like `[output-dir]/base16-[slug][extension]`, where the slug is taken from the scheme filename made lowercase with spaces replaced with dashes and both the extension and output-dir are taken from `/template/config.yaml`.

In the case where schemes share the same slug, a builder will overwrite files previously generated from the template. Should this happen, a builder should show warning messages listing the overwritten files.

## Template Variables
A builder should provide the following variables to a template file:
A builder tool MUST provide the following variables, and no others, to the
template files it processes:

- `scheme-name` - obtained from the scheme file
- `scheme-author` - obtained from the scheme file
- `scheme-slug` - obtained from the scheme filename, as described above
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
- `scheme-kind` - either "light" or "dark", calculated from the `base00` color <!-- TODO: This is a candidate for inclusion, let me know your thoughts -->
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it must have described the computation algorhythm for fallback values in older themes - otherwise it's again doesn't meet the promise of backwar-compatibility

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh? I'm not sure I follow. I think we need to define what "backwards compatible" means... can you give examples of what you think we'd be "breaking" with these changes and how?

Copy link
Member

@actionless actionless Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

means:

  1. if scheme written for previous version of spec:
    1.1) so it doesnt have scheme-kind field defined
  2. and if template written for the current version of spec
    2.1) so it have scheme-kind field used
  3. then builder must compute fallback value for scheme-kind field in such case
    3.1) so spec must define the logic (algorithm) for such computation

Copy link
Member

@actionless actionless Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

actually i've just noticed it mentions what fallback values would be computed, it just had different wording so i didn't noticed it before, although it still doesn't describe the algorithm in the spec

i think the formula should be:
"light" if (base00_r+base00_g+base00_b) > (base07_r+base07_g+base07_b) else "dark"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

calculated from the base00 color

Yes, I assume the spec would specify how this happens mathematically - no argument... there are existing luminescence formulas we should probably use.

Copy link
Member

@actionless actionless Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there are existing luminescence formulas

in theory - yes, but on practice that would en-complicate the spec and so builder implementation notably, whilst r, g, b color computation is already defined in builder spec

and for such comparison the precision of rgb-based color-math is enough

it wouldn't be enough in case of implementing some color transformations a-la base9 or color-mixing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we're talking about light or dark we should use a known luminescence formula - not reinvent the wheel. I do not understand why you think using a common, known luminescence calculation would complicate the spec.


- `base00-hex` to `base0F-hex` - obtained from the scheme file e.g "7cafc2"
- `base00-hex-r` to `base0F-hex-r` - built from the hex value in the scheme file e.g "7c"
- `base00-hex-g` to `base0F-hex-g` - built from the hex value in the scheme file e.g "af"
- `base00-hex-b` to `base0F-hex-b` - built from the hex value in the scheme file e.g "c2"
- `base00-hex-bgr` to `base0F-hex-bgr` - built from a reversed version of all the hex values e.g "c2af7c"

- `base00-rgb-r` to `base0F-rgb-r` - converted from the hex value in the scheme file e.g "124"
- `base00-rgb-g` to `base0F-rgb-g` - converted from the hex value in the scheme file e.g "175"
- `base00-rgb-b` to `base0F-rgb-b` - converted from the hex value in the scheme file e.g "194"
- `base00-dec-r` to `base0F-dec-r` - converted from the rgb value in the scheme file e.g "0.87..."
- `base00-dec-g` to `base0F-dec-g` - converted from the rgb value in the scheme file e.g "0.50..."
- `base00-dec-b` to `base0F-dec-b` - converted from the rgb value in the scheme file e.g "0.21..."

Builders should also provide the following variables for convenience:
## 3. Interface

- `base00-hex` to `base0F-hex` - obtained from the scheme file e.g "7cafc2"
- `base00-hex-bgr` to `base0F-hex-bgr` - built from a reversed version of all the hex values e.g "c2af7c"
All base16 reference builders MUST provide a single feature: building a
template using 1 or more schemes.

This section (`3`) is not relevant to extended tooling, which will have its own
interface fitting their intended usecases, altough they MAY expose the
reference CLI interface (through a `build` subcommand, for example) and/or a
library, if desired.

It is REQUIRED that this functionality is exposed by one (or both) of the following:

### 3.1. CLI

The binary name SHOULD contain `base16`, but is otherwise left as a choice to
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
the author.

To be fully compliant, a builder CLI interface MUST NOT include any other
belak marked this conversation as resolved.
Show resolved Hide resolved
feature, option, argument, or deviance from the expected interface and
behaviour of the program.

<!-- TODO: For convenience, we make a manpage and set of tests available. All
compliant builders MUST fully conform to these two. -->

The CLI interface MUST work without relying on any network connection.

#### 3.1.1 CLI Arguments and options

```bash
base16 <TEMPLATES-DIRECTORY> <SCHEME-FILE> ...
```

`TEMPLATES-DIRECTORY` being a directory containing a `config.yaml`, as well as at
least one `.mustache` template. These are usually named `templates` on all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this TEMPLATES-DIRECTORY the templates directory within a template repo or the template repo itself? i.e. https://github.com/base16-project/base16-vim or https://github.com/base16-project/base16-vim/tree/main/templates?

Edit: from reading more in the "Output and behaviour" below, it seems this is the template dir (or whatever directory contains the .mustache and .config.yaml files and the config.yaml used a relative path to point to the output dir.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup! That's right. The idea is to make it as easy as possible for CI usage (on most repos it is simply templates), while allowing for more advanced usage (such as keeping a set of templates somewhere and building them).

base16 template repositories.

`SCHEME-FILE` being one or more `.yaml` scheme file. The main source for these
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to allow for passing a directory of schemes as well to allow building all schemes at once.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally/alternatively, we should specify that schemes should be loaded (by default) from the base16-schemes repo.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, yet another comment here - one thing I like about the base16-builder-go is the ability to embed a default set of schemes, but allow overriding it if the user wants to clone the schemes repo. Is that something we want to support?

Copy link
Member Author

@Misterio77 Misterio77 Jun 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to allow for passing a directory of schemes as well to allow building all schemes at once.

I think traversing the directory (should it be recursive or not? is there a max depth? etc) is better handled by the calling shell. That's why SCHEME-FILE accepts multiple files. This allows, for example:

base16 template schemes/*.yaml schemes2/*.yaml
base16 template $(find . -name *.yaml)

This also allows easier implementation: all files that should be opened are known by the program without listing files:

  • TEMPLATES-DIRECTORY/config.yaml
  • Templates files specified by the previous yaml
  • All files specified by SCHEME-FILE

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally/alternatively, we should specify that schemes should be loaded (by default) from the base16-schemes repo.

Sorry, yet another comment here - one thing I like about the base16-builder-go is the ability to embed a default set of schemes, but allow overriding it if the user wants to clone the schemes repo. Is that something we want to support?

Hmm, I'd rather not couple the builders with the repo. This would also require all of them to have git as dependencies.

I think that explicit and agnostic is better than implicit and coupled (at least when we're talking about plumbing), so I'd argue that asking the scripter to git clone first would be preferrable. Would that make sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually just updated this - I recently added a command line flag allowing the user to run in "online mode" which pulls the schemes directly from github (using the tarball endpoint - it was surprisingly easy) and I default to that in the provided Github action, but falls back to the "embedded" schemes.

Copy link
Member

@belak belak Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a possible alternative, what would we think of having a --schemes-dir argument (or something similar) and allowing multiple arguments for which templates to process? Or passing the first argument as the schemes dir and everything after that as templates.

Because we've moved to a more consistent base16-schemes repo, we should be taking advantage of that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote is either a single templates directory or calling the CLI multiple times per template (to build all schemes) since again building tons of templates I see as the edge case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really dislike making plumbing tools interact with git or somehow maintain state, so I'd rather be Unixy and make the reference builders just implement the template spec. That is, just build a template + a number of schemes into a number of themes/configs/outputs.

Copy link
Member Author

@Misterio77 Misterio77 Jul 1, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also on the unixyness part, I think iterating over files on a directory is also not very good when we're talking about plumbing tools. There are much better tools available for that (regular shell globbing, find, etc) that can easily be used to get whichever schemes the caller might prefer.

I don't have a strong opinion about building just 1 or multiple templates, as long as we can do something like:

git clone https://github.com/base16-project/base16-schemes
git clone https://github.com/base16-project/base16-shell
git clone https://github.com/base16-project/base16-vim

base16 --templates base16-{shell,vim}/templates --schemes base16-schemes/*.yaml

In this example the builder would (pseudocode):

for template in templates
    subtemplates = read_and_parse(template/config.yaml)
    for subtemplate in subtemplates
        for scheme in schemes
            build(subtemplate, scheme)

So, no directory listing, no download stuff, etc.

is the [base16-schemes](https://github.com/base16-project/base16-schemes)
repository.

The program MAY also offer the following options:

- `--debug | -d` option(s) for increasing log verbosity;
- `--version | -v` option to output its current running version;
- `--help | -h` option to output usage information directly or by opening the
manpage.

These three options have output that is considered implementation detail, not
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
intended to be scripted with. Thus each author SHOULD implement them as they
see fit.

#### 3.1.2. CLI output and behaviour

**Note**: All text outputted by the CLI binary is considered implementation
detail, so the author MAY output whatever they prefer (or no message at all).
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
If needed, scripts using the builder SHOULD check for return codes (specified
below) instead of messages.

For all templates defined in the template config file (`config.yaml`, inside
the specified `TEMPLATES-DIRECTORY`), the builder MUST iterate through all the
specified schemes and output matching files.

The built filenames MUST be relative to the directory where the command is
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly off-topic (and another change to add to the list), but what would you think of changing the filename output to an embedded mustache template, defaulting to reading those variables from the config.yaml and the <output-dir>/base16-<slug><extension> which currently works?

Essentially, replacing all those separate variables with a mustache template and providing a method of backwards compatibility (for now).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I understood. Where would these variables be?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Currently, extension and output-dir live in the template repo's templates/config.yaml. We could add an additional variable called filename or output-file, defaulting to a mustache template: {{ output-dir }}/base16-{{ scheme-slug }}{{ extension }}, and allowing users to change the full filename.

executed, and MUST look like `<output-dir>/base16-<slug><extension>`, where the
`SLUG` is taken from the scheme filename made lowercase with spaces replaced
with dashes and both the extension and `output-dir` are taken from
`config.yaml`.

If the build fails for any reasons, the program MUST exit with code `1` and MAY
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
output an error message. This is not exhaustive and new exit codes might be
added in the future to account for specific errors, so scripters catching
generic errors SHOULD check for non-zero status.

Otherwise, the program MUST exit with code `0` and an OPTIONAL success message.

#### 3.1.3 CLI example:

```bash
git clone https://github.com/base16-project/base16-vim.git
git clone https://github.com/base16-project/base16-schemes.git
base16 base16-vim/templates base16-schemes/*.yaml
```

Should create a `colors` (on the current directory) containing `base16-XXX.vim`
files.

### 3.2. Library

The other option is exposing a software library other developers may use to
assist developing more complex base16-compatible tooling.

As above, the library MUST implement single feature: building templates.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about having another variable here for dark/light theme? I saw that vscode requires this in their templates and the way to deal with this at the moment would be to have a script that checks for "-light." for each colorscheme name and then add the value to the colorschemes.

Maybe this is something that should stay on the template repos side, I just thought I'd bring it up.

Copy link
Member Author

@Misterio77 Misterio77 Jun 25, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a great idea!

If we're asking people to refactor their builders, I don't think it's too much to ask of them. It's actually something I expose with nix-colors (which is basically a nix base16 builder).

We could perhaps specify the strategy used to determine the scheme kind as well, perhaps something simple like: add the three color components together and anything above 382 is dark?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, because all the schemes are in 1 repo, we could add dark: true to relevant schemes and default to dark: false (or the reverse).

This exposed library, or any internal code, has no specific required structure.

The author MAY choose how (through a class, function that operates on file
paths, etc) they will expose the builder functionality to the caller, according
to their judgement.

They SHOULD try to achieve an ergonomical and powerful interface and follow the
best practices on their respective programming languages.

Official reference implementations will follow semantic versioning matching
this spec, in the format: `<base16-version>-builder_revision`.

## Code Structure
There is no outline for a recommended code structure that a base16 theme builder should follow but a design goal should be to keep the application as simple as possible providing only the functionality described in this document. If you feel you have a great idea for additional functionality please raise an issue in the [base16 repository](https://github.com/chriskempson/base16).
## 4. Considerations
Mustache was chosen as the templating language due to its simplicity and
widespread adoption across languages. YAML was chosen to describe scheme and
configuration files for the same reasons.

## Considerations
Mustache was chosen as the templating language due to its simplicity and widespread adoption across languages. YAML was chosen to describe scheme and configuration files for the same reasons.
The core scheme repository was based off the single scheme repository so
builders supporting v0.8-v0.9 of the spec can continue to function without
changes.

The core scheme repository was based off the single scheme repository so builders supporting v0.8-v0.9 of the spec can continue to function without changes.
The revised builder functionality specification introduced in v0.11 did not
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I wouldn't include this line. Maybe it would make sense to move the changelog from the README into this file?

Additionally, maybe it's worthwhile putting our unwritten rule into writing that "spec upgrades should be backwards compatible for templates".

Copy link
Member Author

@Misterio77 Misterio77 Jun 27, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove the 0.11 line but keep the 0.8 and 0.9 one? Or remove both?

introduce any changes to schemes or templates, so no changes to these are
needed. Older builders will continue to work, but authors are encouraged to
implement the new, simpler and more consistent, specification.