Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tracking issue: require(esm) #52697

Open
8 of 9 tasks
joyeecheung opened this issue Apr 25, 2024 · 19 comments
Open
8 of 9 tasks

Tracking issue: require(esm) #52697

joyeecheung opened this issue Apr 25, 2024 · 19 comments
Labels
esm Issues and PRs related to the ECMAScript Modules implementation. module Issues and PRs related to the module subsystem.

Comments

@joyeecheung
Copy link
Member

joyeecheung commented Apr 25, 2024

Before it's unflagged

  • Figure out default export interop with transpilers, either adding __esModule to required ESM on our end (module: add __esModule to require()'d ESM #52166), or transpilers update themselves to check the result returned by require():
  • conditional exports for module, regardless of whether it's loaded by require or import. Something like module which is recognized by Webpack and Rollup would be good (maybe this doesn't need to block unflagging, but should be done before stablization) module: implement the "module-sync" exports condition #54648
  • Move experimental warning to where require() is actually handling a ESM

Before it is promoted to be stable:

Nice-to-haves:

Bug fixes & changes:

Related features that interoperate with require(esm) and need to be considered when being backported together:

For v22.x backport (see a summary of regression analysis in #55217 (comment))

For v20.x backport: TBD

@joyeecheung joyeecheung added module Issues and PRs related to the module subsystem. esm Issues and PRs related to the ECMAScript Modules implementation. labels Apr 25, 2024
@GeoffreyBooth
Copy link
Member

@nodejs/loaders

@jakebailey
Copy link

I just wanted to note it here, but it would be super super awesome if (once stable) this were backported to Node 20/22 or even Node 18 if still in support. I'd love to be able to propose a change to switch TypeScript to ESM (given I have it working without breaking CJS consumers), but the time horizon of Node 22 being the oldest supported version is pretty daunting.

It also seems like there is a hacky way using multiple entrypoints that could allow for TS to grab Node's builtins conditionally without #52599/#52762, though none of that is possible without require(ESM), of course.

Even without TypeScript's use case, I think the feature itself is a really important one for the ecosystem. Backporting would really make ESM changeovers a lot less painful.

@Andarist
Copy link
Contributor

Andarist commented May 7, 2024

IIRC from some Twitter threads - there is a plan to backport this once the feature stabilizes.

@joyeecheung
Copy link
Member Author

joyeecheung commented Jul 11, 2024

Regarding the conditional exports, @guybedford suggested to implement just the module condition that Rollup and Webpack already recognize. I have a WIP but need to do some testing which involves many combinations of test cases 😵‍💫 but will also do some npm crawling to see if it can be picked up/is in conflict with any existing popular packages.

@GeoffreyBooth
Copy link
Member

@guybedford suggested to implement just the module condition that Rollup and Webpack already recognize.

The module top level field, or within exports?

Personally I think require-module makes more sense, as it would complement the existing require and import keys that Node supports.

@joyeecheung
Copy link
Member Author

joyeecheung commented Aug 29, 2024

Opened PR for "module" in #54648

Personally I think require-module makes more sense, as it would complement the existing require and import keys that Node supports.

If we are starting from scratch, yes, but then the "module" condition has already been adopted by bundlers that support require(esm) in the wild, so it seems better to go along with the existing convention. See https://gist.github.com/sokra/e032a0f17c1721c71cfced6f14516c62

nodejs-github-bot pushed a commit that referenced this issue Sep 25, 2024
This patch implements a "module-sync" exports condition
for packages to supply a sycnrhonous ES module to the
Node.js module loader, no matter it's being required
or imported. This is similar to the "module" condition
that bundlers have been using to support `require(esm)`
in Node.js, and allows dual-package authors to opt into
ESM-first only newer versions of Node.js that supports
require(esm) while avoiding the dual-package hazard.

```json
{
  "type": "module",
  "exports": {
    "node": {
      // On new version of Node.js, both require() and import get
      // the ESM version
      "module-sync": "./index.js",
      // On older version of Node.js, where "module" and
      // require(esm) are not supported, use the transpiled CJS version
      // to avoid dual-package hazard. Library authors can decide
      // to drop support for older versions of Node.js when they think
      // it's time.
      "default": "./dist/index.cjs"
    },
    // On any other environment, use the ESM version.
    "default": "./index.js"
  }
}
```

We end up implementing a condition with a different name
instead of reusing "module", because existing code in the
ecosystem using the "module" condition sometimes also expect
the module resolution for these ESM files to work in CJS
style, which is supported by bundlers, but the native
Node.js loader has intentionally made ESM resolution
different from CJS resolution (e.g. forbidding `import
'./noext'` or `import './directory'`), so it would be
semver-major to implement a `"module"` condition
without implementing the forbidden ESM resolution rules.
For now, this just implments a new condition as semver-minor
so it can be backported to older LTS.

Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages
PR-URL: #54648
Fixes: #52173
Refs: https://github.com/joyeecheung/test-module-condition
Refs: #52697
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
nodejs-github-bot pushed a commit that referenced this issue Sep 26, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: #55085
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Oct 1, 2024
This patch implements a "module-sync" exports condition
for packages to supply a sycnrhonous ES module to the
Node.js module loader, no matter it's being required
or imported. This is similar to the "module" condition
that bundlers have been using to support `require(esm)`
in Node.js, and allows dual-package authors to opt into
ESM-first only newer versions of Node.js that supports
require(esm) while avoiding the dual-package hazard.

```json
{
  "type": "module",
  "exports": {
    "node": {
      // On new version of Node.js, both require() and import get
      // the ESM version
      "module-sync": "./index.js",
      // On older version of Node.js, where "module" and
      // require(esm) are not supported, use the transpiled CJS version
      // to avoid dual-package hazard. Library authors can decide
      // to drop support for older versions of Node.js when they think
      // it's time.
      "default": "./dist/index.cjs"
    },
    // On any other environment, use the ESM version.
    "default": "./index.js"
  }
}
```

We end up implementing a condition with a different name
instead of reusing "module", because existing code in the
ecosystem using the "module" condition sometimes also expect
the module resolution for these ESM files to work in CJS
style, which is supported by bundlers, but the native
Node.js loader has intentionally made ESM resolution
different from CJS resolution (e.g. forbidding `import
'./noext'` or `import './directory'`), so it would be
semver-major to implement a `"module"` condition
without implementing the forbidden ESM resolution rules.
For now, this just implments a new condition as semver-minor
so it can be backported to older LTS.

Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages
PR-URL: nodejs#54648
Fixes: nodejs#52173
Refs: https://github.com/joyeecheung/test-module-condition
Refs: nodejs#52697
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Oct 1, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: nodejs#55085
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
@voxpelli
Copy link

voxpelli commented Oct 3, 2024

Raised a question on Twitter to @joyeecheung on my conclusions in https://github.com/voxpelli/investigation-esm-require where it seems like Node 22.9.0 may unintentionally allow some ESM-files to be loaded even without the flag: https://twitter.com/voxpelli/status/1841818608713826693

Mentioning here for sake of completeness, if deemed a correct observation a proper issue will be created

@voxpelli
Copy link

voxpelli commented Oct 4, 2024

The above resulted in a PR to fix it: #55250

targos pushed a commit that referenced this issue Oct 4, 2024
This patch implements a "module-sync" exports condition
for packages to supply a sycnrhonous ES module to the
Node.js module loader, no matter it's being required
or imported. This is similar to the "module" condition
that bundlers have been using to support `require(esm)`
in Node.js, and allows dual-package authors to opt into
ESM-first only newer versions of Node.js that supports
require(esm) while avoiding the dual-package hazard.

```json
{
  "type": "module",
  "exports": {
    "node": {
      // On new version of Node.js, both require() and import get
      // the ESM version
      "module-sync": "./index.js",
      // On older version of Node.js, where "module" and
      // require(esm) are not supported, use the transpiled CJS version
      // to avoid dual-package hazard. Library authors can decide
      // to drop support for older versions of Node.js when they think
      // it's time.
      "default": "./dist/index.cjs"
    },
    // On any other environment, use the ESM version.
    "default": "./index.js"
  }
}
```

We end up implementing a condition with a different name
instead of reusing "module", because existing code in the
ecosystem using the "module" condition sometimes also expect
the module resolution for these ESM files to work in CJS
style, which is supported by bundlers, but the native
Node.js loader has intentionally made ESM resolution
different from CJS resolution (e.g. forbidding `import
'./noext'` or `import './directory'`), so it would be
semver-major to implement a `"module"` condition
without implementing the forbidden ESM resolution rules.
For now, this just implments a new condition as semver-minor
so it can be backported to older LTS.

Refs: https://webpack.js.org/guides/package-exports/#target-environment-independent-packages
PR-URL: #54648
Fixes: #52173
Refs: https://github.com/joyeecheung/test-module-condition
Refs: #52697
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Jan Krems <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@targos
Copy link
Member

targos commented Oct 5, 2024

It looks like npm itself triggers the warning:

$ make lint
cd tools/eslint && if [ -x ""/Users/mzasso/git/nodejs/node/node"" ] && [ -e ""/Users/mzasso/git/nodejs/node/node"" ]; then ""/Users/mzasso/git/nodejs/node/node"" /Users/mzasso/git/nodejs/node/./deps/npm/bin/npm-cli.js ci; elif [ -x `command -v node` ] && [ -e `command -v node` ] && [ `command -v node` ]; then `command -v node` /Users/mzasso/git/nodejs/node/./deps/npm/bin/npm-cli.js ci; else echo "No available node, cannot run \"node /Users/mzasso/git/nodejs/node/./deps/npm/bin/npm-cli.js ci\""; exit 1; fi;
npm warn cli npm v10.8.3 does not support Node.js v23.0.0-pre. This version of npm supports the following node versions: `^18.17.0 || >=20.5.0`. You can find the latest version at https://nodejs.org/.
(node:45192) ExperimentalWarning: Support for loading ES Module in require() is an experimental feature and might change at any time
    at emitExperimentalWarning (node:internal/util:269:11)
    at loadESMFromCJS (node:internal/modules/cjs/loader:1388:5)
    at Module._compile (node:internal/modules/cjs/loader:1517:5)
    at Module._extensions..js (node:internal/modules/cjs/loader:1672:16)
    at Module.load (node:internal/modules/cjs/loader:1328:32)
    at Module._load (node:internal/modules/cjs/loader:1138:12)
    at TracingChannel.traceSync (node:diagnostics_channel:315:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:218:24)
    at Module.require (node:internal/modules/cjs/loader:1350:12)
    at require (node:internal/modules/helpers:138:16)

added 185 packages, and audited 186 packages in 8s

41 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities
Running JS linter...

@joyeecheung
Copy link
Member Author

joyeecheung commented Oct 5, 2024

Yes, because of debug, which I covered in #55217 (comment) and addressed in the last commit of the backport PR. In the last TSC meeting we agreed to silence the warning when require() comes from node_modules on v22 and keep it on v23 to help people notice and update them (with #55241)

joyeecheung added a commit to joyeecheung/node that referenced this issue Oct 7, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: nodejs#55085
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Oct 8, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: nodejs#55085
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Oct 9, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: nodejs#55085
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
tpoisseau pushed a commit to tpoisseau/node that referenced this issue Nov 21, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: nodejs#55085
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
tpoisseau pushed a commit to tpoisseau/node that referenced this issue Nov 21, 2024
When a ESM module cannot be loaded by require due to the presence
of TLA, its module status would be stopped at kInstantiated. In
this case, when it's imported again, we should allow it to be
evaluated asynchronously, as it's also a common pattern for users
to retry with dynamic import when require fails.

PR-URL: nodejs#55502
Fixes: nodejs#55500
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Chemi Atlow <[email protected]>
tpoisseau pushed a commit to tpoisseau/node that referenced this issue Nov 21, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: nodejs#55520
Fixes: nodejs#55516
Refs: nodejs#52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
@jonkoops
Copy link

I'd volunteer to help out to get the backport for v18.x done, as I find your work incredibly valuable, especially as a package author dealing with the module dichotomy. But I fear that I would lack the required expertise (and time) to get it done. Still, it would be incredibly valuable to the larger JavaScript community.

@voxpelli
Copy link

There’s no feature freeze or such a few months prior to a major losing LTS status? Adding something new that can introduce bugs right before stopping all bug fixing feels wrong

@joyeecheung
Copy link
Member Author

I don't think there is a feature freeze policy, but things are subject to the discretion of the release team (hence 1 mentioned above)

@richardlau
Copy link
Member

Node.js 18 is in the maintenance phase of LTS and the default position of the Release WG is that new features do not land on it:

https://github.com/nodejs/Release?tab=readme-ov-file#release-phases

  • Maintenance - Critical bug fixes and security updates. New features may be added at the discretion of the Release team - typically only in cases where the new feature supports migration to later release lines.

Now "supports migration to later release lines" could reasonably be applied to this feature, but it would have to be balanced against risk and personally I'd need a lot more convincing to allow it on Node.js 18 given differences in module loading (several things that are on main/23 are not in Node.js 18 -- if there's uncertainty/trickiness in backporting to 20, backporting to 18 is going to be even harder/riskier).

FWIW the discretion part was added for smaller, self-contained things such as node-addon-api features.

@voxpelli
Copy link

If things go well in v22.x, and we manage to backport it properly to v20.x, we'll unflag it in v20.x in a later release.

And that timeline gives very little time to land anything in v18.x, as it should first be proven in v22.x, then added to v20.x and probably become proven there as well, and only after that would it be up for backport into v18.x, and who knows at which date that ends up, probably at least January or February?

nodejs-github-bot pushed a commit that referenced this issue Nov 23, 2024
As part of the standard experimental feature graduation
policy, when we unflagged require(esm) we moved the
experimental warning to be emitted when require() is
actually used to load ESM, which previously was an error.
However, some packages in the ecosystem have already
being using try-catch to load require(esm) to e.g.
resolve optional dependency, and emitting warning from
there instead of throwing directly could break the CLI
output.

To reduce the disruption for releases, as a compromise, this
patch skips the warning if require(esm) comes from
node_modules, where users typically don't have much control
over the code. This warning will be eventually removed
when require(esm) becomes stable.

This patch was originally intended for the LTS releases,
though it seems there's appetite for it on v23.x as
well so it's re-targeted to the main branch.

PR-URL: #55960
Refs: #55217
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Nov 23, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: nodejs#55085
Backport-PR-URL: nodejs#55217
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Nov 23, 2024
This is faster and more consistent with other places using the
regular expression to detect node_modules.

PR-URL: nodejs#55243
Backport-PR-URL: nodejs#55217
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Refs: nodejs#52697
joyeecheung added a commit to joyeecheung/node that referenced this issue Nov 23, 2024
When a ESM module cannot be loaded by require due to the presence
of TLA, its module status would be stopped at kInstantiated. In
this case, when it's imported again, we should allow it to be
evaluated asynchronously, as it's also a common pattern for users
to retry with dynamic import when require fails.

PR-URL: nodejs#55502
Fixes: nodejs#55500
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Chemi Atlow <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Nov 23, 2024
As part of the standard experimental feature graduation
policy, when we unflagged require(esm) we moved the
experimental warning to be emitted when require() is
actually used to load ESM, which previously was an error.
However, some packages in the ecosystem have already
being using try-catch to load require(esm) to e.g.
resolve optional dependency, and emitting warning from
there instead of throwing directly could break the CLI
output.

To reduce the disruption for releases, as a compromise, this
patch skips the warning if require(esm) comes from
node_modules, where users typically don't have much control
over the code. This warning will be eventually removed
when require(esm) becomes stable.

This patch was originally intended for the LTS releases,
though it seems there's appetite for it on v23.x as
well so it's re-targeted to the main branch.

PR-URL: nodejs#55960
Refs: nodejs#55217
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
joyeecheung added a commit to joyeecheung/node that referenced this issue Nov 23, 2024
As part of the standard experimental feature graduation
policy, when we unflagged require(esm) we moved the
experimental warning to be emitted when require() is
actually used to load ESM, which previously was an error.
However, some packages in the ecosystem have already
being using try-catch to load require(esm) to e.g.
resolve optional dependency, and emitting warning from
there instead of throwing directly could break the CLI
output.

To reduce the disruption for releases, as a compromise, this
patch skips the warning if require(esm) comes from
node_modules, where users typically don't have much control
over the code. This warning will be eventually removed
when require(esm) becomes stable.

This patch was originally intended for the LTS releases,
though it seems there's appetite for it on v23.x as
well so it's re-targeted to the main branch.

PR-URL: nodejs#55960
Refs: nodejs#55217
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Ceres6 pushed a commit to Ceres6/node that referenced this issue Nov 26, 2024
As part of the standard experimental feature graduation
policy, when we unflagged require(esm) we moved the
experimental warning to be emitted when require() is
actually used to load ESM, which previously was an error.
However, some packages in the ecosystem have already
being using try-catch to load require(esm) to e.g.
resolve optional dependency, and emitting warning from
there instead of throwing directly could break the CLI
output.

To reduce the disruption for releases, as a compromise, this
patch skips the warning if require(esm) comes from
node_modules, where users typically don't have much control
over the code. This warning will be eventually removed
when require(esm) becomes stable.

This patch was originally intended for the LTS releases,
though it seems there's appetite for it on v23.x as
well so it's re-targeted to the main branch.

PR-URL: nodejs#55960
Refs: nodejs#55217
Refs: nodejs#52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
aduh95 pushed a commit that referenced this issue Nov 26, 2024
As part of the standard experimental feature graduation
policy, when we unflagged require(esm) we moved the
experimental warning to be emitted when require() is
actually used to load ESM, which previously was an error.
However, some packages in the ecosystem have already
being using try-catch to load require(esm) to e.g.
resolve optional dependency, and emitting warning from
there instead of throwing directly could break the CLI
output.

To reduce the disruption for releases, as a compromise, this
patch skips the warning if require(esm) comes from
node_modules, where users typically don't have much control
over the code. This warning will be eventually removed
when require(esm) becomes stable.

This patch was originally intended for the LTS releases,
though it seems there's appetite for it on v23.x as
well so it's re-targeted to the main branch.

PR-URL: #55960
Refs: #55217
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
This unflags --experimental-require-module so require(esm) can be
used without the flag. For now, when require() actually encounters
an ESM, it will still emit an experimental warning. To opt out
of the feature, --no-experimental-require-module can be used.

There are some tests specifically testing ERR_REQUIRE_ESM. Some
of them are repurposed to test --no-experimental-require-module.
Some of them are modified to just expect loading require(esm) to
work, when it's appropriate.

PR-URL: #55085
Backport-PR-URL: #55217
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: LiviaMedeiros <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Filip Skokan <[email protected]>
Reviewed-By: Michael Dawson <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
PR-URL: #54563
Backport-PR-URL: #55217
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
PR-URL: #55199
Backport-PR-URL: #55217
Reviewed-By: Moshe Atlow <[email protected]>
Reviewed-By: Guy Bedford <[email protected]>
Reviewed-By: Joyee Cheung <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
This is faster and more consistent with other places using the
regular expression to detect node_modules.

PR-URL: #55243
Backport-PR-URL: #55217
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Richard Lau <[email protected]>
Reviewed-By: Marco Ippolito <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
Previously we assumed if `--experimental-detect-module` is true, then
`--experimental-require-module` is true, which isn't the case, as
the two can be enabled/disabled separately. This patch fixes the
checks so `--no-experimental-require-module` is still effective when
`--experimental-detect-module` is enabled.

Drive-by: make the assertion messages more informative and remove
obsolete TODO about allowing TLA in entrypoints handled by
require(esm).

PR-URL: #55250
Backport-PR-URL: #55217
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Reviewed-By: Rafael Gonzaga <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
This previously compiles a script and run it in a new context
to avoid global pollution, which is more complex than necessary
and can be too slow for it to be reused in other cases. The
new implementation just checks the frames in C++ which is safe
from global pollution, faster and simpler.

The previous implementation also had a bug when the call site
is in a ESM, because ESM have URLs as their script names,
which don't start with '/' or '\' and will be skipped. The new
implementation removes the skipping to fix it for ESM.

PR-URL: #55286
Backport-PR-URL: #55217
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
When emitting the experimental warning for `require(esm)`, include
information about the parent module and the module being require()-d
to help users locate and update them.

PR-URL: #55397
Backport-PR-URL: #55217
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Stephen Belanger <[email protected]>
Reviewed-By: Chemi Atlow <[email protected]>
Reviewed-By: James M Snell <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
When a ESM module cannot be loaded by require due to the presence
of TLA, its module status would be stopped at kInstantiated. In
this case, when it's imported again, we should allow it to be
evaluated asynchronously, as it's also a common pattern for users
to retry with dynamic import when require fails.

PR-URL: #55502
Backport-PR-URL: #55217
Fixes: #55500
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Chemi Atlow <[email protected]>
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
Trim off irrelevant internal stack frames for require(esm) warnings
so it's easier to locate where the call comes from when
--trace-warnings is used.

PR-URL: #55496
Backport-PR-URL: #55217
Reviewed-By: Marco Ippolito <[email protected]>
Reviewed-By: Paolo Insogna <[email protected]>
Refs: #52697
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
As part of the standard experimental feature graduation
policy, when we unflagged require(esm) we moved the
experimental warning to be emitted when require() is
actually used to load ESM, which previously was an error.
However, some packages in the ecosystem have already
being using try-catch to load require(esm) to e.g.
resolve optional dependency, and emitting warning from
there instead of throwing directly could break the CLI
output.

To reduce the disruption for releases, as a compromise, this
patch skips the warning if require(esm) comes from
node_modules, where users typically don't have much control
over the code. This warning will be eventually removed
when require(esm) becomes stable.

This patch was originally intended for the LTS releases,
though it seems there's appetite for it on v23.x as
well so it's re-targeted to the main branch.

PR-URL: #55960
Backport-PR-URL: #55217
Refs: #55217
Refs: #52697
Reviewed-By: Matteo Collina <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: #55520
Fixes: #55516
Refs: #52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: #55520
Fixes: #55516
Refs: #52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
ruyadorno pushed a commit that referenced this issue Nov 27, 2024
This tracks the asynchronicity in the ModuleWraps when they turn out to
contain TLA after instantiation, and throw the right error
(ERR_REQUIRE_ASYNC_MODULE) when it's required again. It removes
the freezing of ModuleWraps since it's not meaningful to freeze
this when the rest of the module loader is mutable, and we
can record the asynchronicity in the ModuleWrap right after
compilation after we get a V8 upgrade that contains
v8::Module::HasTopLevelAwait() instead of searching through
the module graph repeatedly which can be slow.

PR-URL: #55520
Fixes: #55516
Refs: #52697
Reviewed-By: Paolo Insogna <[email protected]>
Reviewed-By: Antoine du Hamel <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
Reviewed-By: Jacob Smith <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
esm Issues and PRs related to the ECMAScript Modules implementation. module Issues and PRs related to the module subsystem.
Projects
None yet
Development

No branches or pull requests

8 participants