[p5.js 2.0 RFC Proposal]: Build and test system update

[limzykenneth]

Increasing access

A modern and efficient build and test system significantly reduces the time and compute resource necessary to run build and tests of p5.js. Utilizing current best practice around JavaScript development also enables contributors to get familiar with JavaScript development through working on p5.js.

Which types of changes would be made?

• Breaking change (Add-on libraries or sketches will work differently even if their code stays the same.)
• Systemic change (Many features or contributor workflows will be affected.)
• Overdue change (Modifications will be made that have been desirable for a long time.)
• Unsure (The community can help to determine the type of change.)

Most appropriate sub-area of p5.js?

• Accessibility
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Image
• IO
• Math
• Typography
• Utilities
• WebGL
• Build process
• Unit testing
• Internationalization
• Friendly errors
• Other (specify if possible)

What's the problem?

Current build and test of p5.js has not seen significant updates for awhile and there are several issues that can be improved upon:

• Building and testing currently takes quite a bit of time, up to several minutes on slower machines.
• Many of the toolings used are seeing less development/maintenance.
• Some more advanced build features such as modular build are difficult to support with current system.

What's the solution?

The build system for p5.js will be updated to use Rollup, development server will use Vite, and the test runner will use Vitest.

• Rollup
  • Significantly improves the build speed of the library with the core library taking just a few seconds to build.
  • Have more semantic support of ESM.
  • Simpler config and overall a more modern set of tools.
• Vite
  • Also Vite has a library mode and uses Rollup under the hood for production build, it lacks the flexibility to support complex library build where multiple inputs need to correspond to multiple outputs.
  • Provides a very good development server that can be used to speed up library development. An index.html file is added to the repo for using this development server.
• Vitest
  • Uses the same idea behind Vite's development server to run tests which speeds up tests running significantly.
  • Supports headless browsers, parallel tests running, granular test running in watch mode, and more.
  • While many tests already works, the overall tests will need to have a pass through to correct everything, update to use latest ESM syntax library will be using, and integrate any new test methods where necessary.
  • All tests will run in headless browsers and not test should be run in Node.js as it is not the intended environment for p5.js to run in.

Also see #6716 for some of the systems considered.

To do

In conjuction with refactoring and modularization, the build need to be updated as necessary.

The Vite development server can benefit from a more comprehensive index.html that includes common visual regression cases that contributors can check while they work on the code base.

The majority of pending work will be to update all tests to work with Vitest. The priority being all the existing unit tests. Visual tests may be added after.

Pros (updated based on community comments)

• Much faster development, build, and test cycles
• Better support for latest JavaScript syntax and ecosystem

Cons (updated based on community comments)

• Requires changes to most of the development tooling being used
• Rollup, Vite, and Vitest are still being very actively developed which while it is good that they are receiving good amount of attention, things do change at a faster pace than p5.js itself

Proposal status

Under review

[PimTournaye]

Hi, first time actually trying to contribute to p5 specifically, taking a look at how we might handle this task before diving in.
Edit: also, not trying to jump the queue here, generally interested in how this kind of switch works and evolves over time in a codebase like this

[limzykenneth]

@PimTournaye Sorry for the late reply as I've been working on other stuff at the moment. Also related is that we are not at the stage where we are looking for someone to work on this part just yet, the build system are all setup and running so no work needed while the test will require the refactoring work I'm doing to be completed first. Thanks for you interest anyway, do have a look at other open issues and see if there are anything that you may be able to help with!

[limzykenneth]

Some notes for test related tasks:

• Since we don't need to run visual tests in node.js, we can convert the Vitest workspace config back into a regular Vitest config.
• Thanks to @holomorfo for trying out test coverage. I've looked a bit further into test coverage via Vitest and for best test performance a possible setup is as follows:
  • Switch browser mode provider to playwright.
  • Overcome Playwright's browser install requirement by using @playwright/browser-chromium as a dev dependency.
  • Setup coverage report in Vitest to use v8 provider (much faster than istanbul provider).
  • For CI, it may be possible to also install @playwright/browser-firefox and @playwright/browser-webkit to run tests on all major browser flavors for maximum coverage. Although it will need a slightly more complex setup.
  • Coverage for the full test suite is not showing at the moment maybe due to the size and complex setup of the tests we have, overall test refactor may solve this problem.
• Benchmarking will need to be tested at a later point.