Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update builder specification for v0.11.0 #6

Closed
wants to merge 15 commits into from
Closed

Update builder specification for v0.11.0 #6

Changes from 7 commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
86c933d
Update builder specification for v0.11.0
Misterio77 Jun 22, 2022
c4b5397
info about binary name
Misterio77 Jun 22, 2022
5adbd9f
add info about managers
Misterio77 Jun 22, 2022
a6fea52
specify that no additional variables are allowed, to avoid incompatib…
Misterio77 Jun 22, 2022
4887ee3
fix typo
Misterio77 Jun 25, 2022
5dfd46b
add missing period
Misterio77 Jun 25, 2022
ebbb873
add optional help and version options
Misterio77 Jun 25, 2022
dcf7f47
clarify reference and extended builders, few adjustments and trims
Misterio77 Jun 27, 2022
88c6f14
specify that extended tooling may expose the reference CLI
Misterio77 Jun 27, 2022
701454b
move outputs to CLI section, fix heading levels
Misterio77 Jun 27, 2022
9c73029
small clarification
Misterio77 Jun 27, 2022
b9118e5
move network specification into 3.1 instead of 3.1.1
Misterio77 Jun 27, 2022
413ad81
missing word on 3.2
Misterio77 Jun 27, 2022
4dac926
fix considerations numbering
Misterio77 Jun 27, 2022
ea5f309
apply suggestions
Misterio77 Jun 27, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
146 changes: 119 additions & 27 deletions builder.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,57 +1,149 @@
# Builder Guidelines
**Version 0.10.0**
**Version 0.11.0**

A base16 builder is an application that can build syntax highlighting definition files for text editors by using base16 scheme files which contain a collection of colours and base16 template files which contain syntax highlighting rules.
*The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD",
"SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be
interpreted as described in [RFC 2119](https://datatracker.ietf.org/doc/html/rfc2119).*

Builders are designed for theme maintainers' ease of use. Theme maintainers should provide built versions of their theme so the end user doesn't need to be aware of the builder.
## Introduction

## File Structure
A base16 builder is, essentially, an application that can build base16
templates with base16 [schemes](./styling.md).

### Schemes Repository
A template is a "blueprint" that specifies a file representing how to use the
16 base16 colors with that specific software/format. For example: a
[colorscheme template for vim](https://github.com/base16-project/base16-vim).

The schemes repo should either be stored in a common location (perhaps referred to by environment variable or command line flag) or dynamically embedded in the builder.
A scheme is a color palette that consists of 16 colors (hence the name). For
example: [the solarized scheme](https://github.com/base16-project/base16-schemes/blob/main/solarized-dark.yaml)

- `/*.yaml`
Builders are designed for lower-level ("plumbing") usage, specifically for
scripting and as component to build more complex base16 applications.

### Template Repository
The more complex apps designed for end users ("porcelain") are usually referred
to as **managers**, and they don't need to follow any standard besides helping
with base16 theming somehow, in a way that makes sense for their intended
usecase.

Each template repository should have a templates folder containing a config.yaml and any needed mustache template files.
Template maintainers SHOULD provide built versions (with all existing scheme)
so the end user doesn't need to be aware of the builder.

- `/templates/*.mustache` - A template file (there may be multiples of these)
- `/templates/config.yaml` - A template configuration file
## Interface

## Workflow
The first thing a builder needs to do is parse all the scheme files from the schemes repository (as defined in the [file guidelines](https://github.com/chriskempson/base16/blob/master/file.md)). All files matching the pattern `*.yaml` should be loaded from the root of the schemes repository.
All base16 builders MUST provide a single feature: building a template using 1 or more schemes.

When building a target template repository, a base16 builder should first clear out any old output. Next, for all templates defined in the template repo's config file (located at `/templates/config.yaml`), the builder should iterate through all the defined schemes and output matching files. The built filename should look like `[output-dir]/base16-[slug][extension]`, where the slug is taken from the scheme filename made lowercase with spaces replaced with dashes and both the extension and output-dir are taken from `/template/config.yaml`.
### CLI

In the case where schemes share the same slug, a builder will overwrite files previously generated from the template. Should this happen, a builder should show warning messages listing the overwritten files.
It is REQUIRED that that this functionality is exposed as binary CLI executable
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does CLI need to be required for builder? I propose we just have one CLI implementation. Other builders just act as a library for different languages. And the CLI implementation can be thought of as a library for bash. And details of what the CLI looks like is a decision for the sole CLI implementation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we can require the builder to expose its functionality as CLI and/or Library.

I'm not sure if a single CLI implementation would be very good. What implementation should be the canonical one? And it's pretty easy to implement a CLI after you have the library, so perhaps it isn't too much of a requirement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure if a single CLI implementation would be very good.

Why?

Could you explain why we need all builder to have a CLI? If it's a CLI and has the same interface, why does a user want to one verse another?
I'm not against specifying what the cli should do in the spec, but just against requiring all builder to have one.

Having only one CLI implementation was a follow up point. If there is reason to have more than one implementation, it still doesn't mean we should require all builders to have a CLI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably should move discussion to #13

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I started that before I saw any of this.

with the exact interface described below.

The binary name SHOULD contain `base16`, but is otherwise left as a choice to
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
the author.

To be fully compliant, a builder CLI interface MUST NOT include any other
belak marked this conversation as resolved.
Show resolved Hide resolved
feature, option, argument, or deviance from the expected interface and
behaviour of the program.

<!-- TODO: For convenience, we make a manpage and set of tests available. All
compliant builders MUST fully conform to these two. -->

The CLI interface MUST work without relying on any network connection.

```bash
base16 TEMPLATES-DIRECTORY SCHEME-FILE ...
```

`TEMPLATES-DIRECTORY` being a directory containing a `config.yaml`, as well as at
least one `.mustache` template. These are usually named `templates` on all
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this TEMPLATES-DIRECTORY the templates directory within a template repo or the template repo itself? i.e. https://github.com/base16-project/base16-vim or https://github.com/base16-project/base16-vim/tree/main/templates?

Edit: from reading more in the "Output and behaviour" below, it seems this is the template dir (or whatever directory contains the .mustache and .config.yaml files and the config.yaml used a relative path to point to the output dir.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yup! That's right. The idea is to make it as easy as possible for CI usage (on most repos it is simply templates), while allowing for more advanced usage (such as keeping a set of templates somewhere and building them).

base16 template repositories.

`SCHEME-FILE` being one or more `.yaml` scheme file. The main source for these
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to allow for passing a directory of schemes as well to allow building all schemes at once.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally/alternatively, we should specify that schemes should be loaded (by default) from the base16-schemes repo.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, yet another comment here - one thing I like about the base16-builder-go is the ability to embed a default set of schemes, but allow overriding it if the user wants to clone the schemes repo. Is that something we want to support?

Copy link
Member Author

@Misterio77 Misterio77 Jun 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd like to allow for passing a directory of schemes as well to allow building all schemes at once.

I think traversing the directory (should it be recursive or not? is there a max depth? etc) is better handled by the calling shell. That's why SCHEME-FILE accepts multiple files. This allows, for example:

base16 template schemes/*.yaml schemes2/*.yaml
base16 template $(find . -name *.yaml)

This also allows easier implementation: all files that should be opened are known by the program without listing files:

  • TEMPLATES-DIRECTORY/config.yaml
  • Templates files specified by the previous yaml
  • All files specified by SCHEME-FILE

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Additionally/alternatively, we should specify that schemes should be loaded (by default) from the base16-schemes repo.

Sorry, yet another comment here - one thing I like about the base16-builder-go is the ability to embed a default set of schemes, but allow overriding it if the user wants to clone the schemes repo. Is that something we want to support?

Hmm, I'd rather not couple the builders with the repo. This would also require all of them to have git as dependencies.

I think that explicit and agnostic is better than implicit and coupled (at least when we're talking about plumbing), so I'd argue that asking the scripter to git clone first would be preferrable. Would that make sense?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually just updated this - I recently added a command line flag allowing the user to run in "online mode" which pulls the schemes directly from github (using the tarball endpoint - it was surprisingly easy) and I default to that in the provided Github action, but falls back to the "embedded" schemes.

Copy link
Member

@belak belak Jul 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As a possible alternative, what would we think of having a --schemes-dir argument (or something similar) and allowing multiple arguments for which templates to process? Or passing the first argument as the schemes dir and everything after that as templates.

Because we've moved to a more consistent base16-schemes repo, we should be taking advantage of that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My vote is either a single templates directory or calling the CLI multiple times per template (to build all schemes) since again building tons of templates I see as the edge case.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I really dislike making plumbing tools interact with git or somehow maintain state, so I'd rather be Unixy and make the reference builders just implement the template spec. That is, just build a template + a number of schemes into a number of themes/configs/outputs.

Copy link
Member Author

@Misterio77 Misterio77 Jul 1, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also on the unixyness part, I think iterating over files on a directory is also not very good when we're talking about plumbing tools. There are much better tools available for that (regular shell globbing, find, etc) that can easily be used to get whichever schemes the caller might prefer.

I don't have a strong opinion about building just 1 or multiple templates, as long as we can do something like:

git clone https://github.com/base16-project/base16-schemes
git clone https://github.com/base16-project/base16-shell
git clone https://github.com/base16-project/base16-vim

base16 --templates base16-{shell,vim}/templates --schemes base16-schemes/*.yaml

In this example the builder would (pseudocode):

for template in templates
    subtemplates = read_and_parse(template/config.yaml)
    for subtemplate in subtemplates
        for scheme in schemes
            build(subtemplate, scheme)

So, no directory listing, no download stuff, etc.

is the [base16-schemes](https://github.com/base16-project/base16-schemes)
repository.

The program MAY also offer the following options:

- `--debug | -d` option(s) for increasing log verbosity;
- `--version | -v` option to output its current running version;
- `--help | -h` option to output usage information directly or by opening the
manpage.

These three options have output that is considered implementation detail, not
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
intended to be scripted with. Thus each author SHOULD implement them as they
see fit.

### Library

The compliant base16 builder software MAY also expose a software library other
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you mean by this? Can you give an example?

Copy link
Member Author

@Misterio77 Misterio77 Jun 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure.

Suppose you created a python builder. It probably has some kind of class for representing schemes or templates, possibly method or functions for color conversion or the template replacement itself.

It could be useful for other python programs, for example, a GUI that uses base16 to build some sort of theme. The creator could avoid writing code to implement the base16 standard (parsing schemes, messing with the template, etc) if there was some kind of library they could just import the functionality from.

If the base16 is already very general purpose, why not expose your classes/functions/methods as a library for other python software to use?

developers may use to assist developing more complex base16-compatible
software.

This exposed library, or any internal code, has no required structure or usage.

The author MAY choose how they will expose these functionalities to the caller,
according to their preferences and SHOULD follow best practices on their
respective programming languages.

It is RECOMMENDED that builders follow semantic versioning for their library
interface.

The author MAY implement additional features that are exposed through the
library, as long as it does not affect the CLI functionality compliance.

## Output and behaviour

**Note**: As the CLI is not intended for usual human usage, all outputted text
messages are considered implementation detail, so the author MAY output
whatever they prefer (or no message at all). If needed, scripts using the
builder SHOULD check for return codes (specified below) instead of messages.

For all templates defined in the template config file (`config.yaml`, inside
the specified template directory), the builder MUST iterate through all the
defined schemes and output matching files.

The built filename should look like [output-dir]/base16-[slug][extension],
where the slug is taken from the scheme filename made lowercase with spaces
replaced with dashes and both the extension and output-dir are taken from
`config.yaml`.

The builder MUST check for the (unusual) case where schemes share the same
Misterio77 marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couldn't (shouldn't?) we test for this in the schemes repo? Have some tests running and include this case? I suppose we should still check for this here because someone could clone and edit the schemes dir while testing out a colourscheme and give duplicate a slug value.

Copy link
Member Author

@Misterio77 Misterio77 Jun 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, we shouldn't have any schemes with colliding slugs (actually, all file names on our repo should already be slugified).

And yup, that's exactly it. It could be useful to error out when someone with a modified scheme collection accidentally refers to 2+ schemes sharing same slug, which would be very hard to debug.

My idea is basically erroring out only when the inputs are the same, the outputs should be overwritten silently; as it is a common usecase to run the command to rebuild when the output already exists, and current builders spam a lot of warnings that are only useful to debug that rare collision problem.

slug, in this case the program MUST exit with code `1` and MAY output an error
message.

If the build fails for whatever other reasons, the program MUST exit with code
`2` and MAY output an error message.

## Template Variables
A builder should provide the following variables to a template file:
A builder MUST provide the following, and no others, variables to the template
files it processes:

- `scheme-name` - obtained from the scheme file
- `scheme-author` - obtained from the scheme file
- `scheme-slug` - obtained from the scheme filename, as described above

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What do you think about having another variable here for dark/light theme? I saw that vscode requires this in their templates and the way to deal with this at the moment would be to have a script that checks for "-light." for each colorscheme name and then add the value to the colorschemes.

Maybe this is something that should stay on the template repos side, I just thought I'd bring it up.

Copy link
Member Author

@Misterio77 Misterio77 Jun 25, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's a great idea!

If we're asking people to refactor their builders, I don't think it's too much to ask of them. It's actually something I expose with nix-colors (which is basically a nix base16 builder).

We could perhaps specify the strategy used to determine the scheme kind as well, perhaps something simple like: add the three color components together and anything above 382 is dark?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternatively, because all the schemes are in 1 repo, we could add dark: true to relevant schemes and default to dark: false (or the reverse).

- `base00-hex` to `base0F-hex` - obtained from the scheme file e.g "7cafc2"
- `base00-hex-r` to `base0F-hex-r` - built from the hex value in the scheme file e.g "7c"
- `base00-hex-g` to `base0F-hex-g` - built from the hex value in the scheme file e.g "af"
- `base00-hex-b` to `base0F-hex-b` - built from the hex value in the scheme file e.g "c2"
- `base00-hex-bgr` to `base0F-hex-bgr` - built from a reversed version of all the hex values e.g "c2af7c"

- `base00-rgb-r` to `base0F-rgb-r` - converted from the hex value in the scheme file e.g "124"
- `base00-rgb-g` to `base0F-rgb-g` - converted from the hex value in the scheme file e.g "175"
- `base00-rgb-b` to `base0F-rgb-b` - converted from the hex value in the scheme file e.g "194"
- `base00-dec-r` to `base0F-dec-r` - converted from the rgb value in the scheme file e.g "0.87..."
- `base00-dec-g` to `base0F-dec-g` - converted from the rgb value in the scheme file e.g "0.50..."
- `base00-dec-b` to `base0F-dec-b` - converted from the rgb value in the scheme file e.g "0.21..."

Builders should also provide the following variables for convenience:

- `base00-hex` to `base0F-hex` - obtained from the scheme file e.g "7cafc2"
- `base00-hex-bgr` to `base0F-hex-bgr` - built from a reversed version of all the hex values e.g "c2af7c"

## Code Structure
There is no outline for a recommended code structure that a base16 theme builder should follow but a design goal should be to keep the application as simple as possible providing only the functionality described in this document. If you feel you have a great idea for additional functionality please raise an issue in the [base16 repository](https://github.com/chriskempson/base16).

## Considerations
Mustache was chosen as the templating language due to its simplicity and widespread adoption across languages. YAML was chosen to describe scheme and configuration files for the same reasons.

The core scheme repository was based off the single scheme repository so builders supporting v0.8-v0.9 of the spec can continue to function without changes.
Mustache was chosen as the templating language due to its simplicity and
widespread adoption across languages. YAML was chosen to describe scheme and
configuration files for the same reasons.

The core scheme repository was based off the single scheme repository so
builders supporting v0.8-v0.9 of the spec can continue to function without
changes.

The revised builder functionality specification introduced in v0.11 did not
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Personally, I wouldn't include this line. Maybe it would make sense to move the changelog from the README into this file?

Additionally, maybe it's worthwhile putting our unwritten rule into writing that "spec upgrades should be backwards compatible for templates".

Copy link
Member Author

@Misterio77 Misterio77 Jun 27, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we remove the 0.11 line but keep the 0.8 and 0.9 one? Or remove both?

introduce any changes to schemes or templates, so no changes to these are
needed. Older builders will continue to work, but authors are encouraged to
implement the new, simpler and more consistent, specification.