feat(build)!: Introduce exports section in package.json

[cpcallen]

The basics

• I validated my changes

The details

Resolves

Preparatory work for #7449.

Proposed Changes

Add exports section to package.json

Add an exports stanza to package.json, enumerating existing entrypoints in a new format. This also formally specifies the legal import paths; node.js, webpack etc. will enforce this and prevent paths other than those explicitly enumerated here from being loaded.

Remove most entrypoint wrappers

Since exports allows us to point each entrypoint at any file, by pointing them directly at the relevant compressed chunk wherever possible we eliminate the need for most of the crufty entrypoint wrappers that we have needed up to this point.

Combine browser and node entrypoint wrappers

Combine scripts/package/browser/index.js (was packaged as dist/browser.js) and scripts/package/node/index.js (was packaged as dist/node.js) into a single, environment-agnostic index.js. Drop code generators from the list of things loaded by this entrypoint.

Simplify core entrypoint wrapper for node.js

Move scripts/package/node/core.js to scripts/package/core-node.js, and have it packaged as dist/core-node.js rather than dist/core.js—without a UMD wrapper, since it will always be loaded as a CJS module.

Improve remaining wrapper imports

• Use 'blockly/core' instead of './core' when importing core into index.js (and similarly for the other chunks it imports) so that it will either load blockly_compressed.js (if in a browser) or dist/core-node.js (if in node.js) automatically.

  • This change is necessary because of the removal of the browser stanza from package.json, which would previously substitute one file for another—e.g., require('./core') would be treated by webpack as meaning require('./core-browser.js') if running in a browser environment.

• Some other minor tweaks to packageUMD calls to produce slightly more sensible-looking UMD wrappers.

Remove bogus .d.ts files from typings/msg

PR #3821 added .d.ts files for every file in msg/json/, but several of these are internal utility files rather than translations, and do not result in a langfile being output by create_messages.py when building langfiles.

Reason for Changes

Adding an exports section:

• Allows us to definitively enumerate the permitted entrypoints for the package, disallowing direct importation of any other files.
• Allows us to change which file each individual entrypoint actually loads without it necessarily being a breaking change.
• Allows us to specify separate ESM entrypoints for subpaths (like blockly/javascript; without exports all subpath entrypoints must be either ESM or CJS and it is not possible to provide both.

Test Coverage

• Passes npn test.
• Doesn't break playground.
• The examples/sample-app works fine when npm linked to this version.
• node loads things as expected

Documentation

Our "getting blockly" documentation should be updated to note that loading https://unpkg.com/blockly (and similar) will load blockly.min.js, which consists of blockly_compressed.js, msg/en.js, blocks_compressed.js and javascript_compressed.js concatenated together. Notably, despite (or perhaps because of) each of these having their own UMD wrapper, the combination works fine as a script but will if (somehow) loaded as a CJS or AMD module only return the exports for the last of those chunks—i.e., the JavaScript generator.

(We could alternatively decide to do something else with the umd: condition for the main (`'.'``) entrypoint.)

Additional Information

Module resolution

Module resolution is ultimately at the discretion of each JS runtime or build tool. Browsers, node.js, Webpack, Closure Compiler, etc. each have their own module resolution and loading implementations. The package.json file was originally for the benefit of npm in its role of installing packages and their dependencies, but now contains additional information (like the browser, exports, main, umd, and unpkg directives) that are honoured to varying degrees by different tools. It is therefore hard to say categorically what file(s) will be loaded by which tools without consulting each tool's documentation (and/or implementation), but amongst tools that have some kind of support for NPM packages, the exports directive appears to be widely and relatively consistently supported.

Here is some of the relevant documentation for tools we expect Blockly developers are likely to use:

• node.js's package entry points and CJS and ESM module resolution algorithms.
• webpack's package.json exports support and module resolution and loaders.
• Closure Compiler's support for modules.

Breaking changes

BREAKING CHANGE: The introduction of an exports stanza means that correctly-behaved tools (node.js, bundlers like webpack, etc.) will only allow importing of the specified entrypoints. Here is the full list of permitted entrypoints that can be imported or required:

• blockly
• blockly/core
• blockly/blocks
• blockly/dart
• blockly/lua
• blockly/javascript
• blockly/php
• blockly/python
• blockly/msg/<lang>, for all supported language codes <lang> (e.g blockly/msg/en, blockly/msg/fr, blockly/msg/de, etc.)

If you previously imported any other paths from the blockly package you will need to update your imports. Here are the most common paths that may have been used, and their correct replacements:

If you previously imported:	Import instead:
blockly/index.js	blockly
blockly/node.js	blockly
blockly/browser.js	blockly
blockly/blockly.min	This file should only be loaded as a <script>.
blockly/core.js	blockly/core
blockly/core-browser.js	blockly/core
blockly/blockly_compressed.js	blockly/core
blockly/blocks.js	blockly/blocks
blockly/blocks_compressed.js	blockly/blocks
blockly/dart.js	blockly/dart
blockly/dart_compressed.js	blockly/dart
blockly/lua.js	blockly/lua
blockly/lua_compressed.js	blockly/lua
blockly/javascript.js	blockly/javascript
blockly/javascript_compressed.js	blockly/javascript
blockly/php.js	blockly/php
blockly/php_compressed.js	blockly/php
blockly/python.js	blockly/python
blockly/python_compressed.js	blockly/python
blockly/msg/en.js	blockly/msg/en

If you are loading Blockly as a script you should continue to load the compiled chunks (blockly_compressed.js, blocks_compressed.js, javascript_compressed.js, etc.) directly, or load blockly.min.js which is just a concatenation of blockly_compressed.js, msg/en.js, blocks_compressed.js, and javascript_generator.js.

(Prior to v11 there were various entrypoint wrappers provided in the form of UMD modules (e.g. blockly.js, javascript.js, etc.) but loading these as as scripts never did anything useful and, as they are no longer needed, they have been deleted.)

BREAKING CHANGE: Prior to v11, importing the main blockly package would import blockly/core, blockly/blocks, blockly/msg/en and blockly/javascript—and additionally, in node.js, also blockly/dart, blockly/lua, blockly/php and blockly/python.

Now the main blockly package entrypoint never loads any of the generators.

This change has been made because of changes to generator exports made in blockly v9.0.0 that make necessary to always separately import generator modules in order to use them.

Note that this change does not affect loading the blockly package via <script src="https://unpkg.com/blockly">; that continues to load to blockly.min.js, which continues to include javascript_compressed.js which (when loaded as a script) makes itself available via Blockly.JavaScript for backwards compatibility.

[cpcallen]

Note to reviewer: the diffs, especially of scripts/gulpfiles/package_tasks.js, may be a bit easier to understand if the PR is reviewed commit-by-commit—but note that there are a number of intermediate changes that were almost entirely overwritten or outright deleted by later commits, so this will perhaps be less efficient.

[rachel-fenichel]

Approved.

Two requests (made in offline conversation):

• Add links to an overview of the resolution strategies used by node and by webpack.
• Test with one or more of the plugins.

[cpcallen]

OK, this has tested correctly with blockly-samples, subject to certain updates to blockly-samples:

• plugins/dev-scripts/config/weback.config.js needs various fixes.
• plugins/*/tests/*.mocha.[jt]s need to be updated to import 'blockly' instead of 'blockly/node', since the latter is no longer a valid entrypoint (and AFAIK has never been advertised as one).

I also want to make two additional tweaks to the 'blockly/core' export in this PR before submitting it.

[cpcallen]

OK, previous commits rebased onto current rc/v11.0.0. This will hopefully fix the tests. 🤞