Ability to run plugins on files produced by previous plugins?

[Sidnioulz]

Hi Drew,

I've been wanting to write a Prettier plugin for Cobalt UI to familiarise myself a bit more with the tool. The rationale behind plugins for Prettier/ESLint is simple: having it pre-written makes it faster for folks to connect their formatter/linter to their tool and it avoids reliance on things like post NPM scripts, so it runs the same when Cobalt is called via Node directly in CI, for instance.

But I'm under the impression plugins can only get the dictionary as an input and produce files as an output.

I know I've recently told you it'd be great to let plugins modify the dictionary or return a new dictionary instead of files, and I now find myself wanting to do the opposite: having plugins consume files as input!

And I'm also under the impression that Cobalt aims to have individual plugins produce individual outputs, rather than chainable plugins that are combined to produce an outcome (like SD does transform -> format -> action).

Am I missing something? And if not, what's your general stance on making plugins chainable and are there previous discussions/ADRs I could read to understand why Cobalt is designed the way it is currently?

Thanks

[drwpow]

That’s a great question, and thanks for asking! No one has really asked before 🙂. There are probably 2 main things to touch on: first the state of the API as it exists today, and second, the future possibilities of what it could be.

Current API

Cobalt’s plugin design is heavily-based on Rollup’s plugin system (and by extension, Vite’s, which uses Rollup and extends it). I wanted to create something harmonious with Rollup/Vite for 2 main reasons:

1. Rollup/Vite have proven to be a powerful build tool capable of transforming any file type and assembling very complex architecture efficiently (i.e. it’s more powerful than we’d ever need)
2. Borrowing an existing API means if you’re already using Rollup/Vite, you already know Cobalt! (can’t please everyone of course, but letting one group of devs reuse their knowledge is much better than zero).

Additionally, it’s a simpler architecture than Style Dictionary, based on the “one plugin” approach being simpler to debug. It’s also easier to predict the output of any given token, given each build target is roughly the responsibility of one plugin (though all plugins can affect the final output, which I’ll get back to). Internally I’ve been viewing this improvement as Style Dictionary borrowing ideas from “first gen” build tools (Grunt, Gulp, etc.), and Cobalt borrowing ideas from “second gen” build tools (webpack, Rollup).

In a discussion, someone asked “why not just make it a Vite plugin, then?”. And to expand on that answer but with more context from your question, Cobalt autogenerates nearly any filetype you’d like it to from a JSON or YAML file. In order to pull that off in Rollup/Vite, there would have to be restrictions (such as loading it from your entry bundle, which you may not need to do in many cases). Plus, I didn’t want to only make a tool for Vite users—any design token tooling needs to be compatible with anything. Hence the standalone CLI. But it could always be made into a Vite (or webpack) plugin in the future with the standalone foundation now set.

But to your point about chaining, it’s also worth pointing out that plugins relate 1:1 with build targets, NOT token types. This is under the assumption that the web target may need colors in a different format from native. Or outputting for WebGL (sRGB Uint8Array) may need different outputs than CSS (Display P3). But in any case, the target is the major concern, and the individual token type transformation takes 2nd seat (though most plugins do have a transform option). This is also trying to design in a way that encourages tokens to exist in their final (ish) form in the tokens manifest, to make governance/consistency easier, as well as debugging easier (and not having to spend time figuring out why your colors across all platforms got messed up because one transformer in a very long chain had a bug). Also, as additional context, this does imply that there should be more tools that help assemble that tokens manifest, and it’s a separate-but-compatible project we’re working on currently.

Future Possibilities

But if token types are 2nd class citizens, then that leaves current Cobalt users faced with the following:

• If an existing plugin has the options they need, great! But if they don’t, devs have to build a plugin from scratch (which seems harder than a transformer).
• Taking the output from 1 plugin and transforming it with another is hard/impossible.
• Applying universal transforms across all build targets is hard/impossible.

To address these needs, that’s also why Cobalt is depending on Rollup’s prior art. Whereas Cobalt’s current plugin API only has one step (transform), Rollup’s plugin API has many (load, transform, buildStart, buildEnd, and more). Each step is an additional opportunity for plugins to transform output, and “chain” in one sense (there are some nuances with how Rollup’s resolver works and “claims” files, but I’ll skip over that). Cobalt could add all these steps (and probably should!).

And the fact that you’re asking about it probably means it’s time to explore adding those, so Cobalt can really start to benefit from this design.

To the “building a plugin from scratch” challenge, I’ve realized Cobalt does need to release a standalone package that just deals with common token operations better (or, at the very least, have more docs on “best practices” per token type). As the plugins have grown and matured, there is quite a bit of copy + paste going on (as well as plugin-sass depending on plugin-css, which isn’t a great design). By providing a better toolkit, another design goal is to make plugin authorship easy.

Feedback

Given all that, would love your thoughts about what’s working (or not) with the current design, and would love honest critique about the potential roadmap! Ultimately I’m trying to solve for the easiest way to manage tokens (as we all are), but it takes input from what people need to really get there. And I’m always happy to answer questions or discuss thinking (and am open to feedback on all points).

[drwpow]

now find myself wanting to do the opposite: having plugins consume files as input!

Oh also forgot to address this specifically. I’m hoping that by adding additional hooks to the plugin API (load, buildStart, buildEnd), this would solve for that. Of course, they’d be virtual files, but still in line with your needs (and if it’s not, would love to hear more). Hopefully it still solves the same end goal (and again, borrowing from prior art on how this can be better, such as not having to read/write from disk which is one of the biggest performance bottlenecks any build system can have).

[Sidnioulz]

Hi Drew!

Apologies for taking so long to reply, I wanted to properly reflect on what you said and come up with a common ground we can use to support this conversation. By the way, great to hear you take inspiration from Rollup, it's my fav bundler (no offense!) :)

I appreciate the value of not distinguishing between formats and actions as that distinction has often been a source of pain for my advanced use cases! But at the same time, what was painful to me was the artificial output restriction placed on formats due to the built-in 'simplifications' (it writes to the FS and reports on the created file for you). So maybe I'm worried that the input and output APIs for Cobalt plugins will create such artificial restrictions too.

Input and output types

And this is where I believe there's an important distinction to make with Rollup: you manipulate both virtual files and POJOs (token trees)¹². My two requests (plugins that input tokens and output tokens; and plugins that input files and output files) would both be solved if the plugin API exposed both concepts on an equal footing. It would even allow for plugins that transform files into tokens³.

Having extensively used Rollup, I've never had to figure out how plugins would interface with each other to get a build outcome I wanted. It's always felt natural but also always been a black box, whereas loaders in Webpack made it abundantly clear how most of the pipeline happened. So this is both a strength and a weakness, there needs to be a clear view of the order of plugins matters or if anything is managed through things like enforce and apply. I'm also not so sure if these would be enough to produce a predictable, deterministic order (which I feel token transforms do require). I'll produce a sample pipeline below which we could maybe audit together to see if we feel like Rollup is expressive enough to solve this.

It's easy to map out the 4 scenarios this affords:

• Tokens as inputs and outputs: token transforms
• Tokens as inputs, (virtual) Files as outputs: SD's format equivalent producing an outcome for a platform
• (v)Files as inputs, (v)Files as outputs: last-second concerns like tsc, cjs -> esm, prettier, checksum computation for build caches, cloud upload
• (v)Files as inputs, tokens as outputs: parsers that extract design data from brand assets to help ensure a single source of truth

So I personally feel that we'd get most expressiveness of all these flows of information were possible, and if no specific one was mandated. As you know, I haven't actually tried Cobalt on a large use case yet so I may be simply not understanding it yet and missing the point here.

Branching

Besides that, I've been thinking that pipelines inspired by CI/CD systems would be awesome. With Rollup, we have a single pipeline going on, so if we need to branch out (say, prepare tokens one way except for typography dimension tokens), then we need to run two pipelines. Just like it's common in a Rollup build config to output multiple configs for multiple build targets despite overwhelming similarities between the configs.

Let's look at our current situation:

In the last pipeline I worked on, we had ~25 transforms, ~25 formats and 17 actions and it's still missing a bunch of things. Our formats all carry incredible amounts of complexity internally to handle modes (or contextual values), to support our l10n typography specifics, our themes, and our device specifics. If not, we'd need to run over 400 formats to handle all our use cases.

We run most of the 25ish transforms multiple times because we have to have a pipeline per platform. Sometimes the difference is ridiculously small (e.g. Compose dimensions need a dot between the value and unit, but not Java Android APIs); and we've found it exceedingly rare that the default transform in SD did the work we wanted (though often due to a lack of control on matchers rather than due to the transform logic).

But what if we had a CircleCI or GitLab pipeline? We could have multiple transform plugins run in parallel and save their subtree of the token dictionary to a named resource, and then format plugins load the resources they need (and if those are token trees, Cobalt could merge them on the fly) to produce their output. This would eliminate many duplicate steps and allow us to chain transforms rather than duplicate them when two transforms output something very similar.

Likewise near the end of the pipeline. We have jobs to copy files output by formats and to copy assets (icons/fonts) into our NPM dist, into AWS, into a folder for iOS, Android, etc. We have a prettier job, and we could have a tsc job but I grew tired of SD by the time I had to build TS declarations. Some of the code in these jobs (implemented as SD actions) is near identical, and we could benefit from running them over the entire output once all dependencies have produced their files instead.

So I'm gonna try and find the time to model what a CI pipeline could look like, to see if it seems easier to grasp and reason with than the current monster we have.

Side note on reusability

Just to specifically address this point:

If an existing plugin has the options they need, great! But if they don’t, devs have to build a plugin from scratch (which seems harder than a transformer).

What I mostly felt was that SD often had the transformation/formatting logic we wanted, but it wasn't configurable to match our technical needs. E.g. JS formats that only output to CJS, or iOS formats that output for UIKit but not Swift despite minimal differences. Most transformers doing the right thing but with a CTI matcher that we couldn't use because all our platforms have exceptions (e.g. Android sp vs dp dimensions).

I feel that having a common "interface" for plugins that target an ecosystem might help, e.g. if you provide plugin writers with a helper to add a "cjs/esm" option to their plugin and to convert their plugin output to cjs or esm, then all JS plugins support both formats out of the box. This might be an avenue to explore to avoid the lack of reusability SD currently suffers from.

¹ happy to separately discuss an idea I'm exploring at the moment, where sources of truth should be relational databases and file metadata rather than token trees; there might be interesting design implications if you're willing to entertain the conversation
² corollary: I see less and less value in attributing preset semantics to tokens (e.g. typography, color, etc.) the more I work on DS automation
³ file metadata as source of truth, e.g. we duplicate a lot of our font file info into tokens to correctly build Android/iOS targets, and we have icon metadata that could also be brought into the pipeline

[drwpow]

This is absolute gold! Thank you for taking the time to write all this up! ❤️. I’ll have to spend more time thinking about all your points. I can already think of ways in which we could incorporate big changes to the plugin the API to solve for these, but will have to spend some time on a concrete proposal.

For now I just wanted to add the additional context that I know many workflows resemble your own, powered by SD. And I’ve been interested in answering the question: “Are token pipelines like this because that’s the only pattern SD enables, or would token pipelines be like this regardless of SD because that’s most efficient?” It’s a worthwhile question, and it can only be answered if a viable alternative to SD exists. I don’t think Cobalt in its current state is a complete solution. But usually the first version of anything usually doesn’t get everything right (meaning, both SD as the “first version” of token tooling, and the first version of Cobalt). But as many people are in the same boat as you—only trying out Cobalt rather than building infrastructure around it—Cobalt has the luxury of getting to be more experimental than SD can, to see if there is more to be gained from another approach (just as Snowpack did :P—no offense taken—and it influenced/inspired Vite). And a better path forward is always thanks to thoughtful insights from smart people like yourself 🙂

[Sidnioulz]

I'll do my best to model alternative pipelines for this specific case next week, see if it helps us imagine an API to build it :)

[drwpow]

After thinking about it a little more, the major difference between how Rollup works today and a Rollup-esque token workflow could work is resolveId behavior. In many examples, the source would always be the same, so trying to “claim” an entry file would be a bit silly as every plugin would try (yes some people have multi-source token systems, but that’s an internal detail not a requirement for how the entire system should work).

But I’m thinking that focusing Cobalt’s resolveId around output instead would be an intentional departure that would encourage better interop between plugins (Rollup has a similar concept with virtual files, but often times that’s used in more of an opaque “don’t touch this” way than what we need). I still believe that output should be the major determiner for token operations, not $type. Many times you’re making adjustments per-platform (e.g. what colorspace is supported). And would provide a more exciting exploration for Cobalt while letting SD evolve in the mindset of “per-token” operations.

Open questions, of course, is the whole “module claiming” behavior from Rollup, and the execution order of plugins, which I’m not sure about yet (open to thoughts). Also, having a more “generate files based on output” could be a footgun for accidentally exploding output files (or just being downright confusing) if certain restrictions aren’t put in place.

[Sidnioulz]

But I’m thinking that focusing Cobalt’s resolveId around output instead would be an intentional departure that would encourage better interop between plugins (Rollup has a similar concept with virtual files, but often times that’s used in more of an opaque “don’t touch this” way than what we need). I still believe that output should be the major determiner for token operations, not $type. Many times you’re making adjustments per-platform (e.g. what colorspace is supported). And would provide a more exciting exploration for Cobalt while letting SD evolve in the mindset of “per-token” operations.

Thoroughly agree with this! Having control over how to glue plugins would make it much easier to chain them in ways that hadn't been anticipated by the plugins' designers.

You wouldn't necessarily need to resolve IDs if there's a declarative way to name a plugin's output, so that the same name can be used as input for the next plugin, e.g. like CircleCI build artifacts and workspaces or GitLab job artifacts work. A resolveId hook could be useful if some particular plugins want to be told how to fetch specific data from previous job (e.g. refer to tokens within a JSON input?) but a lighter declaration-based system could also be used. After all, that's what import resolution already does, baked in Rollup, Webpack, etc., through a loose industry standard.

I'm also looking to CI pipelines for job ordering. One thing they do well is to left you declare prerequisites for each step and to then deduce the order of execution and parallelise it for you when possible, leading to better performance. One issue I have with "named steps" is that you may end up not knowing where to position a specific plugin, or you may need to control the order of two plugins on the same step. But an advantage they have is it's easier to provide sane defaults for them. I think that's what GitLab aims to do with stages. You could have a predefined order of stages, and plugins pick a default stage they want to run at, but someone wanting to do multi-platform / complex pipelines can run the same plugin with different inputs/prerequisite plugins and build a more complex graph. Defining prerequisite jobs could cancel out the default stage declaration.

It might also be somewhat easy to build cache on top of a CI-like system where, if the inputs for your plugin and/or the outputs for your required plugins is stable, and it was last run with the same version of your plugin, your plugin might be skippable. Having that baked in would make CD for long token/asset pipelines so much more responsive.

As for UX, I wonder how many DS engineers are comfortable with CI/CD systems vs bundlers like Rollup/Webpack, and how many orgs have people who can intervene on either. My intuition is that more people know CI/CD systems, so it might be harder to get into for folks accustomsed to SD, but more approachable overall. I like your plugin API and I wouldn't change it, but my gut feeling is that the plugin orchestration could take inspiration beyond Rollup.

[drwpow]

Chewed on all your great comments and thoughts, and I’m going to start formulating a 2.0 API for plugins. The DTCG is working on a 1.0 stable release of the spec this year, which may not happen till the very end of the year. So what will probably happen is Cobalt 2.0 might get an early alpha version soon with an enhanced plugin API, but will intentionally hold off on a stable release for a long while, and try to release with support for 1.0 of the stable spec.

---

As an aside, I started scratching out ideas for a completely-different API that is more “pipeline-y” like GitLab and the tools you mentioned. But the more I went down that direction, it seemed to just be a bizarre rework of Style Dictionary’s design which IMO Style Dictionary already does a good job of. But as you pointed out, not all of Rollup’s design makes sense for tokens. So I ended up with some unique hooks that I think will make sense for what we’re doing. But before sharing I want to see if the idea actually works for the existing plugins, and if so, can publish an alpha version (with docs) to get feedback on the new API.

[drwpow]

The 2.0 beta is closer! This issue was a big inspiration for the new plugin API updates. You can also try the in-progress beta with:

pnpm i -D @terrazzo/cli @terrazzo/plugin-css

Would love feedback on the new Plugin API if you have any 🙏. I’m pretty excited about what this unlocks, thanks to your input and feedback!

[Sidnioulz]

Hi Drew!

Yay, super stoked to try this out! I think your hunch was right, my thinking was maybe a bit excessively influenced by SD.

I've since come across interesting use cases which SD is clearly unable to handle (where the name of a token depends on how it's grouped relative to other tokens), where your plugin-based API would outshine it. I'll try and get approval to share that data with you when I'm back in the office. Also got some more complex use cases to try a new API against (cc @pvignau).

At the moment I'm travelling (speaking at Config 😬 odds are you might be there too? 🍻), so I won't give signs of life until August. Will do my best to try it out then!

[drwpow]

At the moment I'm travelling (speaking at Config 😬 odds are you might be there too? 🍻)

Oh that’s awesome, I see you’re speaking on Fri! I’ll also be there (I’m actually working at Figma now) along with some other members of the DTCG spec. Will definitely say “hi” at some point 🍻

[Sidnioulz]

Hi Drew! Damn, I thought I'd answered. If you wanna get in touch I am connected to the Figma slack on #ext-config24-speakers, feel free to say hi :)

[Sidnioulz]

Drew, I've left you a message on slack with my phone number! I'll hanging in the building!