Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: replace ipfs-http-client with kubo-rpc-client #2079

Closed
1 task done
Tracked by #107
SgtPooki opened this issue Jan 12, 2023 · 10 comments · Fixed by #2098
Closed
1 task done
Tracked by #107

feat!: replace ipfs-http-client with kubo-rpc-client #2079

SgtPooki opened this issue Jan 12, 2023 · 10 comments · Fixed by #2098
Assignees
Labels
effort/days Estimated to take multiple days, but less than a week exp/intermediate Prior experience is likely helpful P0 Critical: Tackled by core team ASAP status/ready Ready to be worked topic/dependencies Topic dependencies

Comments

@SgtPooki
Copy link
Member

SgtPooki commented Jan 12, 2023

Is your feature request related to a problem? Please describe.
Since js-kubo-rpc-client is now officially released with all interface tests passing, we should use that instead of ipfs-http-client.

Describe the solution you'd like

  1. Remove ipfs-http-client dependency completely from our dependency tree
  2. Use kubo-rpc-client instead of ipfs-http-client

Describe alternatives you've considered
Discussed elsewhere, please see ipfs/kubo#9125

Additional context


Pre-reqs

Tasks

@SgtPooki SgtPooki added the need/triage Needs initial labeling and prioritization label Jan 12, 2023
@SgtPooki SgtPooki self-assigned this Jan 12, 2023
@SgtPooki SgtPooki added P0 Critical: Tackled by core team ASAP exp/intermediate Prior experience is likely helpful status/ready Ready to be worked topic/dependencies Topic dependencies effort/days Estimated to take multiple days, but less than a week and removed need/triage Needs initial labeling and prioritization labels Jan 12, 2023
@github-project-automation github-project-automation bot moved this to Needs Grooming in IPFS-GUI (PL EngRes) Jan 12, 2023
@SgtPooki SgtPooki moved this from Needs Grooming to Prioritized / Ready for Dev in IPFS-GUI (PL EngRes) Jan 12, 2023
@SgtPooki
Copy link
Member Author

I spent Friday (2023-01-13) attempting this, and we will need to upgrade to ESM first. Things need to be more manageable by doing that first.

@SgtPooki SgtPooki removed their assignment Jan 16, 2023
@SgtPooki SgtPooki changed the title !feat: replace ipfs-http-client with kubo-rpc-client feat!: replace ipfs-http-client with kubo-rpc-client Jan 18, 2023
@tinytb
Copy link

tinytb commented Jan 19, 2023

  • upgrading to ESM should be feasible
  • there will be some complexity in the build tools and dependencies that aren't ESM
  • will have a better idea of what's involved after spending a few hours on this

@SgtPooki
Copy link
Member Author

SgtPooki commented Feb 9, 2023

Currently blocked by ipfs/js-kubo-rpc-client#143 which has a PR out: ipfs/js-kubo-rpc-client#145

@SgtPooki
Copy link
Member Author

this work is mostly done on the esm/kubo-rpc-client branch but but failing in a single e2e test when using ipld-explorer-components (when doing the steps of the test manually, webui works as expected, but confirmed not working when running via playwright)

@SgtPooki
Copy link
Member Author

SgtPooki commented Feb 10, 2023

the ipld-explorer-components code at https://github.com/ipfs/ipld-explorer-components/blob/23cf07da8a78b1d6d9aaa20a8e5a32ef42936ad8/src/bundles/explore.js#L182-L204 is showing undefined for some formats that are returned, which is causing the TypeError: Cannot read properties of undefined (reading 'codec')

image

Trying to debug if ipld-explorer-components is missing a peerDependency callout, or if import is failing because of webui esm importing from the CJS ipld-explorer-components.

funny thing is that this is only occurring in playwright tests though, and works perfectly fine with npm run start. will check if it works with serving the build directly (same way playwright runs the test)

@SgtPooki
Copy link
Member Author

confirmed that when running things manually, it's also broken.

# terminal 1
ipfs daemon

# terminal 2
npm run test:build && npx http-server ./build/ -c-1 -a 127.0.0.1 -p 3001

# browser:
http://127.0.0.1:3001/#/explore/bafkreicgkmwhdunxgdqwqveecdo3wqmgulb4azm6sfnrtvd7g47mnrixji

at least my repro steps should be easier now.

@SgtPooki
Copy link
Member Author

fixed locally with a patch,

diff --git a/node_modules/ipld-explorer-components/dist/bundles/explore.js b/node_modules/ipld-explorer-components/dist/bundles/explore.js
index 123bc7f..8cbbfd1 100644
--- a/node_modules/ipld-explorer-components/dist/bundles/explore.js
+++ b/node_modules/ipld-explorer-components/dist/bundles/explore.js
@@ -388,6 +394,8 @@ function getIpld() {

 function _getIpld() {
   _getIpld = _asyncToGenerator( /*#__PURE__*/regeneratorRuntime.mark(function _callee3() {
+    var _ipldEthereum$default;
+
     var ipldDeps, _ipldDeps$map, _ipldDeps$map2, ipld, formats, ipldEthereum, ipldJson;

     return regeneratorRuntime.wrap(function _callee3$(_context4) {
@@ -405,26 +413,38 @@ function _getIpld() {
             /* webpackChunkName: "ipld" */
             'ipld-git'), import(
             /* webpackChunkName: "ipld" */
-            'ipld-raw'), import(
-            /* webpackChunkName: "ipld" */
-            'ipld-ethereum')]);
+            'ipld-raw')]);

           case 2:
             ipldDeps = _context4.sent;
             // CommonJs exports object is .default when imported ESM style
             _ipldDeps$map = ipldDeps.map(function (mod) {
-              return mod["default"];
+              var _mod$default;
+
+              var actualModule = (_mod$default = mod["default"]) !== null && _mod$default !== void 0 ? _mod$default : mod;
+
+              if (actualModule.name != null && actualModule.code != null && actualModule.codec == null) {
+                actualModule.codec = actualModule.code;
+              }
+
+              return actualModule;
             }), _ipldDeps$map2 = _toArray(_ipldDeps$map), ipld = _ipldDeps$map2[0], formats = _ipldDeps$map2.slice(1); // ipldEthereum is an Object, each key points to a ipld format impl

-            ipldEthereum = formats.pop();
-            formats.push.apply(formats, _toConsumableArray(Object.values(ipldEthereum))); // ipldJson uses the new format, use the conversion tool
+            _context4.next = 6;
+            return import(
+            /* webpackChunkName: "ipld" */
+            'ipld-ethereum');
+
+          case 6:
+            ipldEthereum = _context4.sent;
+            formats.push.apply(formats, _toConsumableArray(Object.values((_ipldEthereum$default = ipldEthereum["default"]) !== null && _ipldEthereum$default !== void 0 ? _ipldEthereum$default : ipldEthereum))); // ipldJson uses the new format, use the conversion tool

-            _context4.next = 8;
+            _context4.next = 10;
             return import(
             /* webpackChunkName: "ipld" */
             '@ipld/dag-json');

-          case 8:
+          case 10:
             ipldJson = _context4.sent;
             formats.push(convert(ipldJson));
             return _context4.abrupt("return", {
@@ -432,7 +452,7 @@ function _getIpld() {
               formats: formats
             });

-          case 11:
+          case 13:
           case "end":
             return _context4.stop();
         }

but display name is still different from explore.ipld.io:

webui local changes

image

explore.ipld.io

image


From debugging this is due to codec numbers being used instead of names at https://github.com/ipfs/ipld-explorer-components/blob/23cf07da8a78b1d6d9aaa20a8e5a32ef42936ad8/src/components/object-info/ObjectInfo.js#L18-L29

working on a patch (and then a PR to ipld-explorer-components)

@SgtPooki
Copy link
Member Author

Fixed block naming with

diff --git a/node_modules/ipld-explorer-components/dist/components/object-info/ObjectInfo.js b/node_modules/ipld-explorer-components/dist/components/object-info/ObjectInfo.js
index c4bcf4e..cd8112e 100644
--- a/node_modules/ipld-explorer-components/dist/components/object-info/ObjectInfo.js
+++ b/node_modules/ipld-explorer-components/dist/components/object-info/ObjectInfo.js
@@ -15,6 +15,7 @@ import { withTranslation } from 'react-i18next';
 import { ObjectInspector, chromeLight } from '@tableflip/react-inspector';
 import filesize from 'filesize';
 import LinksTable from './LinksTable';
+import multicodec from 'multicodec';
 var humansize = filesize.partial({
   round: 0
 });
@@ -77,16 +78,30 @@ var nodeStyles = {
     color: '#383838'
   }
 };
+/**
+ * Support getting the style object for a node type using the codec number by redirecting the number to the name
+ */
+
+var nodeStylesProxy = new Proxy(nodeStyles, {
+  get: function get(target, prop) {
+    if (isNaN(prop)) {
+      return target[prop];
+    }
+
+    console.log("getting codec name from code number for ".concat(prop, ": "), multicodec.getNameFromCode(prop));
+    return target[multicodec.getNameFromCode(prop)];
+  }
+});
 export function shortNameForNode(type) {
-  var style = nodeStyles[type];
+  var style = nodeStylesProxy[type];
   return style && style.shortName || 'DAG';
 }
 export function nameForNode(type) {
-  var style = nodeStyles[type];
+  var style = nodeStylesProxy[type];
   return style && style.name || 'DAG Node';
 }
 export function colorForNode(type) {
-  var style = nodeStyles[type];
+  var style = nodeStylesProxy[type];
   return style && style.color || '#ea5037';
 } // '/a/b' => ['$', '$.a', '$.a.b']
 // See: https://github.com/xyc/react-inspector#api

image

@SgtPooki
Copy link
Member Author

npm run test is succeeding fully again

@SgtPooki
Copy link
Member Author

npm run test-storybook:ci succeeding. rebasing and opening PR

@SgtPooki SgtPooki linked a pull request Feb 10, 2023 that will close this issue
2 tasks
@SgtPooki SgtPooki moved this from In Progress to In Review in IPFS-GUI (PL EngRes) Feb 22, 2023
SgtPooki added a commit that referenced this issue Mar 13, 2023
* feat!: ipfs-http-client -> kubo-rpc-client

fix: error rendering peerId

fix: peers table on peerspage

fix: statusPage -> advanced -> addresses

fix: most e2e tests working with kubo-rpc-client

chore: update to go-ipfsv0.18.1

* chore: fix key.gen with patch

needed until kubo-rpc-client fix is merged

see ipfs/js-kubo-rpc-client#143
see ipfs/js-kubo-rpc-client#145

* fix: use ipld-explorer-components patch temporarily

see https://github.com/ipfs/ipfs-webui/issues/2079\#issuecomment-1426219633
see #2079 (comment)

must use until ipfs/ipld-explorer-components#356

* chore: fix ipld-explorer-components patch

not needed once ipfs/ipld-explorer-components#356 is merged

* test: add tests to explore.test.js

* tmp: trying to navigate the undocumented world of parsing CIDs

* chore: working on getting tests fully tied up

* test(e2e): finish updating explore.test.js

* chore: lint fixes

* chore: remove debugging code

* test: test clicking children links on explore page

* tmp: fix children explore views

* test: e2e explore test traverses children

* Update test/e2e/explore.test.js

Co-authored-by: Nishant Arora <[email protected]>

* chore: address PR comments

---------

Co-authored-by: Nishant Arora <[email protected]>
@github-project-automation github-project-automation bot moved this from In Review to Done in IPFS-GUI (PL EngRes) Mar 13, 2023
ipfs-gui-bot pushed a commit that referenced this issue Apr 24, 2023
## [3.0.0](v2.22.0...v3.0.0) (2023-04-24)

 CID `bafybeic4gops3d3lyrisqku37uio33nvt6fqxvkxihrwlqsuvf76yln4fm`

 ---

### ⚠ BREAKING CHANGES

* migrate to ESM (#2092)

### Features

* ipfs-http-client -> kubo-rpc-client ([#2098](#2098)) ([5e53c79](5e53c79)), closes [#issuecomment-1426219633](https://github.com/ipfs/ipfs-webui/issues/issuecomment-1426219633) [/github.com//issues/2079#issuecomment-1426337490](https://github.com/ipfs//github.com/ipfs/ipfs-webui/issues/2079/issues/issuecomment-1426337490)
* migrate to ESM ([#2092](#2092)) ([58a737c](58a737c))

### Bug Fixes

* e2e/explore.test.js succeeds in offline mode ([#2109](#2109)) ([a5e9ac6](a5e9ac6))
* ko language falls back to ko-KR ([#2102](#2102)) ([3369800](3369800))
* semantic release custom notes import ([#2113](#2113)) ([2f9f306](2f9f306))

### Trivial Changes

* add NetworkTraffic.stories.js ([#2094](#2094)) ([7a3bf46](7a3bf46))
* pull new translations ([#2101](#2101)) ([cbabac3](cbabac3))
* pull new translations ([#2104](#2104)) ([4a691a2](4a691a2))
* Pull transifex translations ([#2088](#2088)) ([a5b8a1c](a5b8a1c))
* Pull transifex translations ([#2091](#2091)) ([d209863](d209863))
* Pull transifex translations ([#2099](#2099)) ([1cf2fe7](1cf2fe7))
* Pull transifex translations ([#2111](#2111)) ([57d9b63](57d9b63))
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
effort/days Estimated to take multiple days, but less than a week exp/intermediate Prior experience is likely helpful P0 Critical: Tackled by core team ASAP status/ready Ready to be worked topic/dependencies Topic dependencies
Projects
No open projects
Status: Done
Development

Successfully merging a pull request may close this issue.

2 participants