synchronize peer and channel data in fiber network

[rabbitz]

Create or Update Fiber Peer

• POST /api/v2/fiber/peers
• request

  {
    "name": "test", # Peer alias
    "rpc_listening_addr": "http://127.0.0.1:8227",  # Peer listening address
    "peer_id": "QmVCtxcWjouWHyfsjzjJCP18YDdfiYfBMMqxKwF9wmB6CT"  # Peer ID
  }

• response
  • If the peer_id validation succeeds, it returns 204 No Content.
  • If the peer_id validation fails, the response is as follows, where detail contains the error message returned by the RPC:

    [
      {
          "title": "fiber peer params invalid",
          "detail": "not support hash code at line 1 column 60",
          "code": 2010,
          "status": 404
      }
    ]

Peers list

• GET /api/v2/fiber/peers
• params
  • page # default is 1
  • page_size # default is 10
• response

  {
     "data": {
         "fiber_peers": [
             {
                 "name": "test",
                 "peer_id": "QmVCtxcWjouWHyfsjzjJCP18YDdfiYfBMMqxKwF9wmB6CT",
                 "rpc_listening_addr": "http://127.0.0.1:8227",
                 "first_channel_opened_at": null,
                 "last_channel_updated_at": null,
                 "channels_count": 1,
                 "total_local_balance": "193800000000.0"
             }
         ]
     },
   "meta": {
           "total": 1,
           "page_size": 10
   }
 }

Peer detail

• GET /api/v2/fiber/peers/{peer_id}
• response

  {
      "data": {
          "peer_id": "QmVCtxcWjouWHyfsjzjJCP18YDdfiYfBMMqxKwF9wmB6CT",
          "rpc_listening_addr": "http://127.0.0.1:8227",
          "first_channel_opened_at": null,
          "last_channel_updated_at": null,
          "fiber_channels": [
            {
                "peer_id": "QmdYroJM6VAmWGqRHa2GCEvxBTUE7Q4eW4gcm38M8foYxy",
                "channel_id": "0xf6c9802ef1baeef8f18c0c15887652e76d97244388771c178e2929cee11c4e41",
                "state_name": "CHANNEL_READY",
                "state_flags": []
            },
            ...
        ]
      }
  }

Channel detail

• GET /api/v2/fiber/channels/{channel_id}
• response

  {
      "data": {
          "channel_id": "0xf74c5362e7b5639a936b67acd1e761b4693a063f3956e35bfc0875aabcf6e96f",
          "state_name": "CHANNEL_READY",
          "state_flags": [],
          "shutdown_at": null,
          "created_at": "2024-09-12T09:22:42.000+08:00",
          "updated_at": "2024-09-19T20:41:23.111+08:00",
          "local_balance": "193800000000.0",
          "offered_tlc_balance": "0.0",
          "remote_balance": "0.0",
          "received_tlc_balance": "0.0"
      }
  }

[Keith-CY]

Need support from the Fiber Network team to add RPCs for retrieving peer list and channel-open/close transaction list.

RPCs are expected to be added in 2 weeks.

[Keith-CY]

Frontend PR: Magickbase/ckb-explorer-frontend#426

[rabbitz]

Graph Nodes

• GET /api/v2/fiber/graph_nodes
• params
  • page # default is 1
  • page_size # default is 10
• response

{
  "data": {
    "fiber_graph_nodes": [
      {
        "alias": "",
        "node_id": "024714ca19abea4ddc0f3863ffdfb2e2cee76af87c477de4bc67c74a83f8140042",
        "addresses": [
          "/ip4/0.0.0.0/tcp/8228/p2p/Qmes1EBD4yNo9Ywkfe6eRw9tG1nVNGLDmMud1xJMsoYFKy",
          "/ip4/54.179.226.154/tcp/8228/p2p/Qmes1EBD4yNo9Ywkfe6eRw9tG1nVNGLDmMud1xJMsoYFKy/p2p/Qmes1EBD4yNo9Ywkfe6eRw9tG1nVNGLDmMud1xJMsoYFKy"
        ],
        "timestamp": "1728541089425",
        "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "auto_accept_min_ckb_funding_amount": ""
      },
      ...
    ]
  },
  "meta": {
    "total": 4,
    "page_size": 10
  }
}

Graph Node Detail

• GET /api/v2/fiber/graph_nodes/{node_id}
• response

{
  "data": {
    "alias": "",
    "node_id": "026ff293a8e0990dc73108e77a7cc09c5f7069b743a11d49b6a7885e03657912a7",
    "addresses": [
      "/ip4/127.0.0.1/tcp/8228/p2p/QmRX6L8LiHSGLyqu6zArXx3FLTtL8KuEdqyB1e1KeqMrrd"
    ],
    "timestamp": "1727683693959",
    "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
    "auto_accept_min_ckb_funding_amount": "",
    "fiber_graph_channels": [
      {
        "channel_outpoint": "0xe29000aff6950883beca7e6c8329dc99afd5044c25547f11959e70f50ca0a06800000000",
        "node1": "026ff293a8e0990dc73108e77a7cc09c5f7069b743a11d49b6a7885e03657912a7",
        "node2": "02849f365a83aea567de017075402d78fadbce6af6828fbaf6b0a9be1cbc76e460",
        "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
        "funding_tx_block_number": "14746323",
        "funding_tx_index": "0",
        "last_updated_timestamp": "1727683757684",
        "created_timestamp": "1727683757683",
        "node1_to_node2_fee_rate": "1000",
        "node2_to_node1_fee_rate": "1000",
        "capacity": "206200000000.0"
      }
    ]
  }
}

Graph Channels

• GET /api/v2/fiber/graph_channels
• params
  • page # default is 1
  • page_size # default is 10
• response

{
    "data": {
        "fiber_graph_channels": [
            {
                "channel_outpoint": "0xe29000aff6950883beca7e6c8329dc99afd5044c25547f11959e70f50ca0a06800000000",
                "node1": "026ff293a8e0990dc73108e77a7cc09c5f7069b743a11d49b6a7885e03657912a7",
                "node2": "02849f365a83aea567de017075402d78fadbce6af6828fbaf6b0a9be1cbc76e460",
                "chain_hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
                "funding_tx_block_number": "14746323",
                "funding_tx_index": "0",
                "last_updated_timestamp": "1727683757684",
                "created_timestamp": "1727683757683",
                "node1_to_node2_fee_rate": "1000",
                "node2_to_node1_fee_rate": "1000",
                "capacity": "206200000000.0"
            }
        ]
    },
    "meta": {
        "total": 1,
        "page_size": 10
    }
}

[Keith-CY]

Now we have basic data from graph_nodes and graph_channels RPC(https://github.com/nervosnetwork/fiber/tree/main/src/rpc#graph_nodes), following aggregated data can be supported further:

1. total capacity of a graph node: summarize capacity of channels bound to a graph node
2. open/close transaction of a graph channel: each graph channel has its own channel_outpoint, where the funding is. The transaction generates the output is an open transaction, while the transaction consumes it is a close one
3. tokens in a graph channel: udt_cfg_infos is returned in graph node, the token info can be retrieved from the explorer's database according to the type script
4. token in a graph channel: udt_type_script is returned in graph channel, and the token info can be retrieved from the explorer's database according to the type script
5. nodes connected to a graph node: remote nodes connected to a graph node can be aggregated from its graph channels, for now, only the node id/pub key is necessary.

cc @rabbitz

[rabbitz]

1. Graph Node Enhancements:

   • add total_capacity to the node list and details for capacity stats
   • include udt_cfg_infos in the node list and details to show UDT info like full_name, symbol, decimal, and icon_file
   • add connected_node_ids to show connected node IDs

2. Graph Channel Enhancements:

   • add open_transaction_hash for the opening transaction
   • include udt_cfg_info in the channel details to show UDT information

[Keith-CY]

1. Graph Node Enhancements:

   • add total_capacity to the node list and details for capacity stats
   • include udt_cfg_infos in the node list and details to show UDT info like full_name, symbol, decimal, and icon_file
   • add connected_node_ids to show connected node IDs

2. Graph Channel Enhancements:

   • add open_transaction_hash for the opening transaction
   • include udt_cfg_info in the channel details to show UDT information

total capacity, udt info and connected nodes have been added.

I found that open transaction of a channel can be located, while a channel belongs to a node, so we can have a open transaction list of a node

It should cover open transaction of all open/closed channels, and can be sorted by time. Following fields of a transaction are required:

• tx hash
• block number
• timestamp
• funding amount(ckb and udt)
• funding address
• transaction fee

---

Speaking of channels of a node, I'm not sure if closed channels are listed in the response. I hope we can get the list of closed channel, either by the current API or a new API.

---

For the graph nodes API, I hope a new field that count of open channels can be added

---

To display the share of each node, e.g. capacity/udt_amount of node / total capacity/udt_amount in fiber network, I hope there could be an API of amount of various assets, e.g.

{
  "capacity": "100000",
  "udts": [
    {
      "type_hash": "type hash of RUST",
      "decimal": "8",
      "amount": "1000000",
      "symbol": "RUST"
    }
  ]
}

@rabbitz

[rabbitz]

1. Graph Node Enhancements:

   • add open_channels_count to the node list
   • in node details, replace open_transaction_hash with outpoint_info in channel data:

   "outpoint_info": {
       "tx_hash": "0xb5044450a8f93f3b78b4dda13eb4a60210107c67e6de780f990d6e1cae1c953c",
       "block_number": 14983532,
       "block_timestamp": 1729582163892,
       "transaction_fee": 920,
       "funding_capacity": "106200000000.0",
       "funding_udt_amount": null,
       "funding_address": "ckt1qpkx0zrlugq7urrc20ckstqtwlqwvg2qgnq4d364sf5njz52lfkhcqfc6qhc4l5luvc5k0ma2ts8f826mjyqqhc0p2l7w"
   }

   • add status param filter to node details (open by default; returns closed channels when set to closed)

2. Graph Channel Enhancements:

   • add outpoint_info to the channel list

3. every 5 minutes, check open_channels; if the funding_cell is consumed, mark the channel as closed.

[Keith-CY]

Feature request from Cell Studio

1. Support searching fiber: channel id, node id, node name, pubkey, rpc address

cc @rabbitz

[rabbitz]

Feature request from Cell Studio

1. Support searching fiber: channel id, node id, node name, pubkey, rpc address

cc @rabbitz

1. Added filter parameter q to the graph node list, supporting exact matches on alias, peer_id, and node_id.
2. /api/v1/suggest_queries now supports graph node quires.

Issues:

1. graph channel does not return the channel_id, so filtering is not supported.
2. node_id is the public key; filter by node_id is now supported.

rabbitz/ckb-explorer@76d5f19

[Keith-CY]

Feature request from Cell Studio

1. Support searching fiber: channel id, node id, node name, pubkey, rpc address

cc @rabbitz

1. Added filter parameter q to the graph node list, supporting exact matches on alias, peer_id, and node_id.
2. /api/v1/suggest_queries now supports graph node quires.

Issues:

1. graph channel does not return the channel_id, so filtering is not supported.
2. node_id is the public key; filter by node_id is now supported.

rabbitz/ckb-explorer@76d5f19

Searching with

await fetch('https://ckb-explorer-api-staging.magickbase.com/api/v1/suggest_queries?q=02095434545011ead9bf736205cd7d08e9bea87b948a4792d10e127360bb761270',
{
  headers: {
    "Content-Type": "application/vnd.api+json",
    "accept": "application/vnd.api+json"
  }
}).then(res => res.json())

returns not found.
q is node id of https://ckb-explorer-frontend-in-magickbase-repo-git-5ad97f-magickbase.vercel.app/fiber/graph/node/02095434545011ead9bf736205cd7d08e9bea87b948a4792d10e127360bb761270
also tried with 0x prefix