-
Notifications
You must be signed in to change notification settings - Fork 231
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #484 from ipfs/feat/routing-v1-protocols-param
IPIP-484: Opt-in Filtering on Routing V1 HTTP API
- Loading branch information
Showing
2 changed files
with
158 additions
and
5 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,106 @@ | ||
| --- | ||
| title: 'IPIP-0484: Opt-in Filtering in Routing V1 HTTP API' | ||
| date: 2024-10-29 | ||
| ipip: ratified | ||
| editors: | ||
| - name: Daniel Norman | ||
| github: 2color | ||
| affiliation: | ||
| name: Shipyard | ||
| url: https://ipshipyard.com | ||
| relatedIssues: | ||
| - https://github.com/ipfs/ipfs-check/issues/70 | ||
| - https://github.com/ipfs/boxo/pull/678 | ||
| order: 484 | ||
| tags: ['ipips'] | ||
| --- | ||
|
|
||
| ## Summary | ||
|
|
||
| Add opt-in support for filtering specific network transports and/or transfer protocols to the Delegated Routing v1 HTTP endpoints via HTTP GET parameters. | ||
|
|
||
| ## Motivation | ||
|
|
||
| IPFS aims to allow ubiquitous data exchange across different runtimes and platforms. One of the most challenging aspects of this goal is the diversity of network conditions and capabilities across different environments. Web browsers have a very limited network stack, and most web browsers do not support the full range of network transport protocols that are commonly used in other IPFS implementations. | ||
|
|
||
| The Delegated Routing v1 API empowers resource constrained clients like web browsers by significantly reducing the number of network connections necessary to fetch content addressed data directly from provider peers. | ||
|
|
||
| However, there are many cases where most of the results from the Delegated Routing v1 API are not actionable by clients, because the client does not support either the **network transport protocol** or the **transfer protocol** of the provider. | ||
|
|
||
| For instance, web browsers are limited to a specific set of network transport protocols, namely HTTPS, Secure WebSockets, WebTransport (emerging), and WebRTC. Consequently, providing information about providers that exclusively support TCP and/or UDP is not beneficial for browser-based clients, as they are unable to utilize such connections. | ||
|
|
||
| Moreover, [Helia](https://github.com/ipfs/helia/), the most actively maintained browser IPFS implementation, supports block retrieval by CID with Bitswap and Trustless Gateways, but does not support Graphsync. | ||
|
|
||
| This means that returning providers that only support raw TCP, raw UDP/QUIC, or Graphsync from the Delegated Routing API is not useful for browser clients, and results in unnecessary network traffic for browser clients. | ||
|
|
||
| ## Note on terminology | ||
|
|
||
| The term **"transport"** is overloaded in the IPFS ecosystem. | ||
|
|
||
| In the context of this IPIP, we refer to the network layer transport protocol, e.g. TCP, QUIC, WebTransport, as **"network transport protocol"** to avoid ambiguity. | ||
|
|
||
| **"Transfer protocol"** refers to data transfer protocols, i.e. content-addressed block retrieval protocols, e.g. Bitswap, Graphsync, HTTP. | ||
|
|
||
| ## Detailed design | ||
|
|
||
| ### Network Address Filtering | ||
|
|
||
| The proposed change is to add a `?filter-addrs` parameter to the `GET /routing/v1/providers/{cid}` and `GET /routing/v1/peers/{peer-id}` endpoints of :cite[http-routing-v1]: | ||
|
|
||
| - Add a `?filter-addrs=<comma-separated-list>` optional parameter to `GET /routing/v1/providers/{CID}` that indicates which network transports to return by filtering the multiaddrs in the `Addrs` field of the [Peer schema]. | ||
| - The value of the `filter-addrs` parameter is a comma-separated list of network transport protocol _name strings_ as defined in the [multiaddr protocol registry](https://github.com/multiformats/multiaddr/blob/master/protocols.csv), e.g. `?filter-addrs=webtransport`. | ||
| - `unknown` can be be passed to include providers whose multiaddrs are unknown, e.g. `?filter-addrs=unknown`. This allows filtering providers whose multiaddrs are unknown at the time of filtering. | ||
| - Multiaddrs are filtered by checking if the protocol name appears in any of the multiaddrs (logical OR). | ||
| - Negative filtering is done by prefixing the protocol name with `!`, e.g. to skip IPv6 and QUIC addrs: `?filter-addrs=!ip6,!quic-v1`. Note that negative filtering is done by checking if the protocol name does not appear in any of the multiaddrs (logical AND). | ||
| - If no parameter is passed, the default behavior is to return the original list of addresses unchanged. | ||
| - If only negative filters are provided, addresses not passing any of the negative filters are included. | ||
| - If positive filters are provided, only addresses passing at least one positive filter (and no negative filters) are included. | ||
| - If both positive and negative filters are provided, the address must pass all negative filters and at least one positive filter to be included. | ||
| - If there are no multiaddrs that match the passed transports, the provider is omitted from the response. | ||
| - Filtering is case-insensitive. | ||
|
|
||
| ### IPFS Protocol Filtering | ||
|
|
||
| The proposed change is to add a `?filter-protocols` parameter to the `GET /routing/v1/providers/{cid}` and `GET /routing/v1/peers/{peer-id}` endpoints of :cite[http-routing-v1]: | ||
|
|
||
| - Add a `?filter-protocols=<comma-separated-list>` optional parameter to `GET /routing/v1/providers/{CID}` to filter providers based on the `Protocol` field of the [Peer schema]. | ||
| - The `filter-protocols` parameter is a comma-separated list of transfer protocol names, e.g. `?filter-protocols=transport-bitswap`. | ||
| - Transfer protocols names should be treated as opaque strings and have a max length of 63 characters. A non-exhaustive list of transfer protocols are defined per convention in the [multicodec registry](https://github.com/multiformats/multicodec/blob/3b7b52deb31481790bc4bae984d8675bda4e0c82/table.csv#L149-L151). | ||
| - Implementations MUST preserve all transfer protocol names when returning a positive result that matches one or more of them. | ||
| - A special `unknown` name can be be passed to include providers whose transfer protocol list is empty (unknown), e.g. `?filter-protocols=unknown`. This allows for including providers returned from the DHT that do not contain explicit transfer protocol information. | ||
| - Providers are filtered by checking if the transfer protocol name appears in the `Protocols` array (logical OR). | ||
| - If the provider doesn't match any of the passed transfer protocols, the provider is omitted from the response. | ||
| - If a provider passes the filter, it is returned unchanged, i.e. the full set of protocols is returned including protocols that not included in the filter. (note that this is different from `filter-addrs` where only the multiaddrs that pass the filter are returned) | ||
| - Filtering is case-insensitive. | ||
| - If no parameter is passed, the default behavior is to not filter by transfer protocol. | ||
|
|
||
| :::note | ||
| Even though some of existing IPFS transfer protocol names start with `transport`, e.g. `transport-bitswap`, `transport-graphsync-filecoinv1`, and `transport-ipfs-gateway-http`, they should not to be confused with the network transport protocols used in peer addresses, which are filtered using the `filter-addrs` parameter. | ||
| ::: | ||
|
|
||
| ## Design rationale | ||
|
|
||
| - Using these query parameters improves cache efficiency, as the response will be smaller and more specific to the client's needs. | ||
| - Backward compatibility is maintained by not changing the default behavior of the API. | ||
| - Use of protocol name rather than codes makes it easier for human debugging. | ||
| - DHT providers currently do not contain any transfer protocol information. `unknown` can be passed to `filter-protocols` to include such providers. | ||
| - Since provider records are independent of peer records, and it's pretty common to have provider records without up-to-date multiaddrs for that peer, `unknown` can be passed to `filter-addrs` to include such providers. | ||
| - Combining transfer protocol and transport protocol filters is done by ANDing the results of the filters, e.g. `?filter-addrs=webtransport&filter-protocols=transport-bitswap` will return providers that support bitswap and have a webtransport multiaddr. | ||
|
|
||
| ### User benefit | ||
|
|
||
| By filtering out providers that do not support the desired network transport protocol and/or transfer protocol, the client can reduce the traffic necessary in order to fetch the data. | ||
|
|
||
| Moreover, it makes it much easier to determine whether there are any browser-usable providers for a given CID, which is a common use case for clients. | ||
|
|
||
| ### Compatibility | ||
|
|
||
| This should not effect existing clients or servers. | ||
|
|
||
| The default behavior when `?filter-addrs` and `?filter-protocols` is not passed is left unspecified, this IPIP is limited to opt-in behavior. | ||
|
|
||
| ### Copyright | ||
|
|
||
| Copyright and related rights waived via [CC0](https://creativecommons.org/publicdomain/zero/1.0/). | ||
|
|
||
| [Peer schema]: https://specs.ipfs.tech/routing/http-routing-v1/#peer-schema |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters