[p5.js 2.0 RFC Proposal]: Functional core and modular build

[limzykenneth]

Increasing access

p5.js as a library is mostly about the easy to use syntax to enable drawing onto a virtual canvas that it provides its users, the underlying implementation can and should change depending on what the specific needs of the environment are. With a functional core that additional modules can be attached onto, p5.js can be made maximally flexible across environment and diverse user needs.

Which types of changes would be made?

• Breaking change (Add-on libraries or sketches will work differently even if their code stays the same.)
• Systemic change (Many features or contributor workflows will be affected.)
• Overdue change (Modifications will be made that have been desirable for a long time.)
• Unsure (The community can help to determine the type of change.)

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

What's the problem?

The way p5.js is currently split into modules utilizes ES6 modules side-effect imports which on the one hand is not very semantic, and on the other hand more significantly, different modules depends on each other to a large extent, making it hard to exclude certain module where it should not have been needed.

The current build of p5.js library files are also very big. Being able to modularize the build to not include features the user may not need can reduce file size significantly.

What's the solution?

p5.js will have a core that contains only the absolute essential functionalities, while all other code will be separately import-able. Take as example the core only build of p5.js and the math module as a separate module not included in core, two versions of each will be built from the source code: IIFE and ESM. Rollup is setup to create build for both with IIFE being the preferred format for most users using <script> tags and ESM the preferred format for users using their own bundlers. (CommonJS or AMD format will not be supported)

IIFE

Immediately Invoked Function Expression (IIFE) is a common format libraries meant to be included with regular <script> tag will come in. It prevents excessive global namespace pollution and is also used by p5.js 1.0. Rollup have several output formats and libraries meant to be included with regular <script> tag will either be using iife or umd, the later of which combines IIFE, CommonJS, and RequireJS/AMD module syntax in one. p5.js 2.0 will mainly use Rollup's iife format as IIFE allows for initialization through just side effects while for UMD, an export name must be set which is not compatible with the IIFE usage we want.

<script src="./lib/p5.js"></script>
<script src="./lib/p5.math.js"></script>
<script>
function setup(){
  createCanvas(400, 400);
  console.log(ceil(2.1));
}

function draw(){
  background(200);
  circle(200, 200, 100);
}
</script>

In the above example, both p5.js and p5.math.js are built with the iife format. This usage is similar if not identical to the usage of addon libraries currently (deliberately so). The math module here is a bundled module taken from all the source located in src/math folder. Each file in that folder can be independenly built into the iife format with Rollup if desired and the whole module can be included in the final p5.js bundle if desired as well.

// src/math/index.js
import calculation from './calculation.js';
import noise from './noise.js';
import random from './random.js';
import trigonometry from './trigonometry.js';
import math from './math.js';
import vector from './p5.Vector.js';

export default function(p5, fn){
  p5.registerAddon(calculation);
  p5.registerAddon(noise);
  p5.registerAddon(random);
  p5.registerAddon(trigonometry);
  p5.registerAddon(math);
  p5.registerAddon(vector);
}

// src/math/calculation.js (redacted)
function calculation(p5, fn){
  fn.abs = Math.abs;
}

export default calculation;

if(typeof p5 !== 'undefined'){
  calculation(p5, p5.prototype);
}

// src/app.js (redacted)
// math (include if to be bundled as part of p5.js)
import './math/calculation';
import './math/math';
import './math/noise';
import './math/p5.Vector';
import './math/random';
import './math/trigonometry';

Please see relevant examples in the exploration fork for implementation.

ESM

ESM or ES Module is the current standard in JavaScript for working with modular JavaScript code. ESM are now very widely supported with all major browsers natively supporting it, all modern build tools supports or are even built around it, and Node.js have native support for it as well. p5.js 1.0's code is already written with ESM and transpiled into a UMD module. As part of the refactor mentioned in a previos section, the syntax of the internal use of ESM will be updated to match semantic usage. The main goal will be to limit cross dependencies between modules and minimize the use of side effects imports.

import p5 from 'p5';
import math from 'p5/math';

p5.registerAddon(math);

// The same instance mode syntax
const sketch = (p => {
  p.setup = () => {
    p.createCanvas(400, 400);
    console.log(p.ceil(2.1));
  };

  p.draw = () => {
    p.background(200);
    p.circle(200, 200, 100);
  };
});

new p5(sketch);

The above example assumes the user is using Node.js module resolution and have installed p5 through NPM. However, distributable ESM modules are built and will be published via CDN as well. To use this, the first two lines will instead be:

import p5 from './js/p5.esm.js';
import math from './js/p5.math.esm.js';

The registerAddon static method is further described in the library proposal and represent a unified way that internal module and third party library extends the functionalities of the p5.js core.

Pros (updated based on community comments)

• Potential for modular build with smaller foot print
• Unified interface for authoring libraries and working on internal modules, if you are a library author, you will also know how to work on p5.js internals

Cons (updated based on community comments)

• Require significant refactoring of code base
• May require some internal functionalities to be redesigned/rethought, although these should not require user facing breaking changes

Proposal status

Under review

[mattdesl]

I think modularisation would be a huge benefit to v2 but I worry about the approach of registerAddon as it feels a little clunky and adds a degree of boilerplate to each sketch.

It is a drastic change but I would rather see something like the example below, as it is (1) tree-shakeable, (2) does not require referencing or polluting a single p5 instance with non-core parts, (3) is simpler IMHO to reason about and build apps/libraries on top of. However I understand it may not be possible given the wide range of features p5 supports and how many of them will be inter-dependent on each other, and it's also a departure from the simplicity of having everything under a single namespace.

import p5 from 'p5';
import * as math from 'p5/math.js';

// The same instance mode syntax
const sketch = (p => {
  p.setup = () => {
    p.createCanvas(400, 400);
    console.log(math.ceil(2.1)); // math is being used directly
  };

  p.draw = () => {
    p.background(200);
    p.circle(200, 200, 100);
  };
});

new p5(sketch);

[limzykenneth]

@mattdesl The current idea I have may end up supporting both cases actually, at least for some modules. If #6830 is implemented, there should not be a big difference between directly importing say the math module and using it with p5.js here, and using the math module externally without p5.js. Some modules may still need to be explicitly linked to p5 in some way because it may reference something available in the core internally.

The registerAddon interface will likely still be implemented as part of #7015 just so that authoring third party addons that can extend p5.js itself (instead of being entirely external) can be easier. The internal modules using the same interface is mainly to keep consistency and enable transition from addon library author to core library contributor smoother.

[mvicky2592]

I agree with @mattdesl. Modules shouldn't use the same system as libraries, it's clunky and too much boilerplate code. I think Modules should be conceptually separate from libraries. Modules should just simply add functionality to p5, they don't need to use the method hooks that libraries do.

Consider that even the named import syntax is intimidating for beginners and explaining it requires they already know how objects work.

import * as math from 'p5/math.js';

Even though having imports be loaded through side effects is frowned upon by professional devs, perhaps it'd be best if p5 modules were just added to p5 behind the scenes to keep things simple on the user end.

import 'p5/math.js';

Modules could just be added like this:

p5.modules.math = function (p5Instance) {
  // add math functions here
};

And p5.js would load them like this:

for (let module of modules) {
  module(this);
}

That way p5.js could be compatible with q5.js modules.

What do you think? @limzykenneth

[davepagurek]

Modules shouldn't use the same system as libraries, it's clunky and too much boilerplate code.

Are you mostly referring to the boilerplate for library devs or for the end user?

Modules should just simply add functionality to p5, they don't need to use the method hooks that libraries do.

To add some context for the current rationale for this, it's about making it easy to add core-like features in third party addons. This would mean people aren't blocked on GitHub review if they want to try out building a feature that they want to work like a core feature (e.g. building something like an SVG renderer), and it'd make it easy for us to package it along with core builds later on. Maybe not the most standard thing to optimize for, but something based on our current bottlenecks. This isn't set in stone yet of course.

Even though having imports be loaded through side effects is frowned upon by professional devs, perhaps it'd be best if p5 modules were just added to p5 behind the scenes to keep things simple on the user end.

I think that's kind of how the proposal mentions the core modules are getting loaded right now in app.js. @limzykenneth does this mean that if users want to load addons, they could also just do import 'some-p5-addon'; and it would auto set itself up as long as it's imported after p5 and the module file calls p5.registerAddon(...)?

Modules could just be added like this:

This is kinda similar to what the proposal is describing I think. Starting from your idea, it looks like the current proposal makes these changes:

• It wraps the whole thing in if(typeof p5 !== 'undefined') { ... } so that it could still be loaded independently of p5 and manually be set up if need be
• Instead of just passing in p5Instance as a parameter, it also passes in the prototype as another parameter, and lifecycle hooks as a third parameter. I think these could all be accessed through the instance if you need them, so the extra params are like shortcuts (is that right @limzykenneth?) and can just be ignored if you want.
• It uses a different way of attaching to p5 if it exists (p5.registerAddon(yourAddon) rather than p5.modules.yourAddon = yourAddon.) Are there benefits to having an object with all the modules like that rather than just setting them up immediately?

[mvicky2592]

It wraps the whole thing in if(typeof p5 !== 'undefined') { ... } so that it could still be loaded independently of p5 and manually be set up if need be

@davepagurek ah okay I missed that part.

I can see how @limzykenneth's src/math/calculation.js supports side effect loading but I can't see how it could support the use case @mattdesl suggested. The export would just be a function. Manually setting it up would require the user to run that function with an object and the object's prototype as inputs. Then create a new instance of that object math to be able to use it.

import * as mathModuleFn from 'p5/math.js';
let MyMath = {};
mathModuleFn(MyMath, MyMath.prototype);
let math = new MyMath();

Only then would math.ceil work.

Instead of just passing in p5Instance as a parameter...

In limzy's user code, the p5.registerAddon function is run before new p5() so it can't pass a p5 instance, it passes p5 and p5.prototype. Not being able to pass p5 instances to modules would certainly differentiate modules from library hooks, but it seems like an unnecessary limitation.

Modules that just add functionality to a p5 instance could hypothetically be loaded when the sketch actually needs the module's features. I'm not sold on the real world purpose for providing this but it'd be easier for users this way. edit: Also overriding p5 methods could be done on an instance basis.

// p5/math.js
if (typeof p5 == undefined) p5 = { modules: {} };
p5.modules.math = function (p5Instance) {
  p5Instance ??= {};
  // ...
  return p5Instance;
}

let math = p5.modules.math();

Are there benefits to having an object (p5.modules) with all the modules like that rather than just setting them up immediately?

I imagine that some modules will depend on others being loaded. A module could also provide altered functionality depending on which modules are loaded. Having all the modules be added to p5.modules would enable module devs to access that info.