How do I use an npm package in my contract / dapp?

[dckc]

Suppose I want to use moment or lodash or the like in a contract? Is there any trick to it? Any gotchas to watch out for?

something about workspaces...

patches...

[dckc]

yarn add

The normal thing is

$ yarn add xyz
...edit your code to import / use xyz
$ yarn test  # to confirm that it works

workspaces

Yes, dapp-offer-up and such use yarn workspaces.

  "workspaces": [
    "contract",
    "ui"
  ],

https://github.com/Agoric/dapp-offer-up/blob/b61d1025fb01097954cda159c5b65786a878fb74/package.json#L7

So if this is a contract dependency, cd contract before yarn add. If it's a ui thing, cd ui.

Note: they share a lockfile. Sometimes that's wonderful, sometimes not.

[dckc]

Troubleshooting SES_IMPORT_REJECTED

SES_IMPORT_REJECTED explains why they're not allowed.

IOU info about how to trouble-shoot and what to do about it... for example, how to patch with patch-package or yarn 4's built-in support for patching.

[dckc]

troubleshooting lodash?

I'm not sure what's up here.

REJECTED from ava test("Start the contract"): (TypeError#2)
TypeError#2: Cannot read properties of undefined (reading 'Array')
  at runInContext (eval at <anonymous> (eval at makeEvaluate (file:///tmp/aug28/node_modules/ses/src/make-evaluate.js:92:27)), <anonymous>:1452:25)
  at eval (eval at <anonymous> (eval at makeEvaluate (file:///tmp/aug28/node_modules/ses/src/make-evaluate.js:92:27)), <anonymous>:17182:11)
  at eval (eval at <anonymous> (eval at makeEvaluate (file:///tmp/aug28/node_modules/ses/src/make-evaluate.js:92:27)), <anonymous>:17209:3)
  at Object.execute (.../compartment-mapper/src/parse-pre-cjs.js:40:1)
  at eval (.../aug28-contract/src/offer-up.contract.js:1:177)
  at async importBundle (.../import-bundle/src/index.js:57:18)

All I did was yarn add lodash and then import it into the contract and make trivial use of it.

diff --git a/contract/package.json b/contract/package.json
index a9db991..7929b38 100644
--- a/contract/package.json
+++ b/contract/package.json
@@ -50,7 +50,8 @@
     "@agoric/zoe": "^0.26.3-u12.0",
     "@endo/far": "^0.2.22",
     "@endo/marshal": "^0.8.9",
-    "@endo/patterns": "^0.2.5"
+    "@endo/patterns": "^0.2.5",
+    "lodash": "^4.17.21"
   },

diff --git a/contract/src/offer-up.contract.js b/contract/src/offer-up.contract.js
index e24f7d7..b0b11c6 100644
--- a/contract/src/offer-up.contract.js
+++ b/contract/src/offer-up.contract.js
@@ -25,6 +25,9 @@ import { AssetKind } from '@agoric/ertp/src/amountMath.js';
 import { AmountShape } from '@agoric/ertp/src/typeGuards.js';
 import { atomicRearrange } from '@agoric/zoe/src/contractSupport/atomicTransfer.js';
 import '@agoric/zoe/exported.js';
+import _ from 'lodash';
+
+console.log(_.defaults({ 'a': 1 }, { 'a': 3, 'b': 2 }));
 
 const { Fail, quote: q } = assert

[erights]

I figured out a lot about the problem, but not enough to know how to fix it. The problem seems to only happen during bundling which is consistent with your stack trace above. Further, it might have something to do with our bundler's cjs support, but I'm not sure. Attn @kriskowal @naugtur

A separate experiment at endojs/endo#2437 demonstrates that, aside from bundling, lodash and the ses-shim seem to play well together.

It is indeed a problem in the lodash code we're running during bundling, as seen in the debugger:

  /** Detect free variable `global` from Node.js. */
  var freeGlobal = typeof global == 'object' && global && global.Object === Object && global;

  /** Detect free variable `self`. */
  var freeSelf = typeof self == 'object' && self && self.Object === Object && self;

  /** Used as a reference to the global object. */
  var root = freeGlobal || freeSelf || Function('return this')();

Under Hardened JS (including the ses-shim), the Function constructor only does strict code, there is no built-in global or self, and so the code above will set root to undefined, which is consistent with what I saw in the debugger.

But the closest I could find within the lodash sources is already a corrected version that would work instead.
https://github.com/lodash/lodash/blob/main/src/.internal/root.ts
tries to initialize root to the global object in a platform independent way that includes legacy platforms. It does it correctly for modern platforms, including all those of interest to us including the ses-shim itself, because it checks globalThis. The code we're actually running, as seen above, does not.

But even after cloning lodash and grepping, I could not figure out where in the lodash sources the bad version occurs. Googling, I found

https://git.fluig.com/users/csouza/repos/json-server/browse/node_modules/lodash/_freeGlobal.js?at=e658d6d
and
https://git.fluig.com/users/csouza/repos/json-server/browse/node_modules/lodash/_root.js?at=e658d6d
which together reproduce the broken version. But I don't know how that relates to the actual lodash.

I tried changing the package.json dependency from "lodash" to "lodash-es" but it did not seem to help.

Could you please change the title of this issue to be specific to bundling+lodash, so I can link to it from the tracking issue endojs/endo#2037 without confusion? Thanks.

Hopefully, with these clues, someone can pick it up from here. Please?

[erights]

Or, perhaps move this issue to the endo repository, as it seems to have nothing to do with agoric-sdk specifically?

[naugtur]

LavaMoat doesn't have this problem because we emulate global aliases - window, self and in webpack plugin even global (that's because webpack emulates it for us)

I vote we emulate self by default.

Alternative that I'd rather avoid: emulate a return value from Function that when passed 'return this' it actually creates a ()=>compartment.globalThis instead of trying to evaluate anything.

[dckc]

Could you please change the title of this issue to be specific to bundling+lodash

I can make a new one.

I already did narrow #9958 from a general question to one about @noble/hashes but in office hours, realized that it does make sense to document the general situation too: what's the normal way to add dependencies to a dapp? Stuff like workspaces makes the answer non-trivial. And a place to collect a list of known things and how to troubleshoot them seems useful.

[dckc]

it seems to have nothing to do with agoric-sdk specifically?

The problem comes up in the flow of:

1. make an Agoric dapp: yarn create @agoric/dapp dapp-foo; cd dapp-foo; yarn
2. add a dependency: cd contract; yarn add pkg-bar; ...

The fix might be in some lower level component, but the agoric-sdk customer definitely suffers from it. So I want to keep something at this level too.

[erights]

I also found https://gist.github.com/Thinkscape/ca9a7e47e3b44a5cc92d8e552f455eb6 , which seems relevant.

[naugtur]

I don't think lodash would be willing to give up that trick, but maybe we could try to persuade them it's no longer needed. Anyway - endo needs to emulate self

[erights]

But lodash already has source code for the correct version, that also checks globalThis. Why isn’t it using that one? Cheers, --MarkM

…

On Thu, Aug 29, 2024 at 1:37 AM Zbyszek Tenerowicz ***@***.***> wrote: I don't think lodash would be willing to give up that trick, but maybe we could try to persuade them it's no longer needed. Anyway - endo needs to emulate self — Reply to this email directly, view it on GitHub <#9986 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AACC3THYJBGOB46NKWXJJDTZT3MU7AVCNFSM6AAAAABNIUVXKWVHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTANBYGQZDQNQ> . You are receiving this because you commented.Message ID: ***@***.***>