-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: advertise http endpoint as extended provider #27
Conversation
64b2e35
to
ecf33b3
Compare
This is a first pass at adding http as provider to our advertisments for the IPNI indexers. The goal is to advertise our content as being available over bitswap and http. **NOTE** This module needs refactoring and updating to the (dag) haus style. I have not tackled that yet. This PR is currently makes only the minimal changes required to add Extended Providers. The logic of how to serialise and sign a Provider was derivied from the exiting signing code in this module and the implementation in go-libipni see: https://github.com/ipni/go-libipni/blob/afe2d8ea45b86c2a22f756ee521741c8f99675e5/ingest/schema/envelope.go#L125-L167 License: MIT Signed-off-by: Oli Evans <[email protected]>
ecf33b3
to
feed5f0
Compare
src/handlers/advertisement.js
Outdated
async function providerSignature (previous, peerId, cid, contextId, provider, extendedProviderOverride = false) { | ||
const sigBuf = Buffer.concat([ | ||
previous ? Buffer.from(CID.parse(previous['/']).bytes) : Buffer.alloc(0), | ||
Buffer.from(cid.bytes), | ||
Buffer.from(peerId.toString(), 'utf-8'), | ||
contextId, | ||
Buffer.from(provider.ID, 'utf-8'), | ||
...provider.Addresses.map(a => Buffer.from(a, 'utf-8')), | ||
provider.Metadata, | ||
extendedProviderOverride ? Buffer.from([1]) : Buffer.from([0]) | ||
]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The spec for this says
- The
Signature
for each of theProviders
within anExtendedProvider
is signed by their
corresponding private key.
* The full advertisement object is serialized, with all instances ofSignature
replaced with
an empty array of bytes.
* This serialization is then hashed, and the hash is then signed.
* TheProvider
from the encapsulating advertisement must be present in theProviders
of
theExtendedProvider
object, and must sign in this way as well. It may omitMetadata
andAddresses
if they match the values already set at the encapsulating advertisement.
However,Signature
must be present.
– https://github.com/ipni/specs/blob/main/IPNI.md#extendedprovider
In the implementing here I choose to explicitly duplicate all the info from bitswap provider at the advertisment level into the extended provider rather than rely on a fallback to the parent element.
In order to find out what The full advertisement object is serialized
means I had to go to the reference impl:
https://github.com/ipni/go-libipni/blob/afe2d8ea45b86c2a22f756ee521741c8f99675e5/ingest/schema/envelope.go#L125
TODO: figure out if this change is sufficient to get the indexers to apply return the additional http provider for all previous entries. from the spec
...here we're adding extended providers rather than updating the top level provider info, but one would assume that the same logic applies to extended providers. |
There does not appear to be any tests here that check the shape of the advertisements! |
peerId and friends are all a little different now License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🚀
License: MIT Signed-off-by: Oli Evans <[email protected]>
License: MIT Signed-off-by: Oli Evans <[email protected]>
I have
next up i'm deploying the dev branch, and will create some test message in the sqs queue. |
Code is deployed. I have put a test multibase prefixed multihash
{
"head": {
"/": "baguqeera7izo6lfcvowkpqg2jjuo2hgglxy5bfbtwla7i4if6zxuhhe3yu4q"
},
"pubkey": {
"/": {
"bytes": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDuuLoNO5mhRCrLIeKE/zVBX3r3Y9SLbalOtwbYvvh/ounJYq+xatHHCXCx/3OnSDsMfHQxSuDWN8o1YleYPGaoZoLyMy5BvUOMgrRFFbTsiQIbPHo2p+wpRPXSq4wofudz7wBRUdjAPsYkMgUxi/757Z1dpjq+D9FpzwPHqx9nYBGzsKBaqNAcQjb3EbYFrU/xYrl+bYqNCYXgAhXfTtGCKnZAjOiuRHMl2cIy39X50kMfFnXm/gMiVJVf0pxGUsLSfl8HHc0hYLQdYgbSOYBQLamTvzYQJHmfDjgZfmHuuGv+Emh3a/Tq8PNDIgZqf5sAELXQAwIt+Fl/hJK+QgHHAgMBAAE"
}
},
"sig": {
"/": {
"bytes": "fGmvYUeZc0LBt0d2w+F6iQnRBbVaCkGS2OnPR2UmhkOlrY/u6/aH/xqplVWGOcfdQtGFMB89Qsh23AMi0fSimdi0TjtAlY52E/8UNpyLrz5E4DMIW527FnJK8xw+x4si6S41z80b3VvLm5UVEr+KadsN6lXTgFb7q37+Q5PNggtyZ5+uKFVyh/wSlJIK6Q4OjrPXm081EhItLLYSH5H0nADexIrWXb6cc0ILCcMnI+exRkAjehC2UJUA5u6Uu7qMQaL9z6nfn3/P1aQGNowNk1Avr4PBfwpbz4e6g61nvhQ7F32vrw+vvyFCiJs7DhMPXM/d9zeuKqdZsmZQ2NZqZg"
}
}
} The head {
"Addresses": [
"/dns4/elastic-dev.dag.house/tcp/443/wss"
],
"ContextID": {
"/": {
"bytes": "YmFndXFlZXJhamtzMnJxcmQ1N2NlZHl4YW9lYXU3cjdtcmRudDVxbW83dGx5NXlnaHAybnB3YmxqM3NqYQ"
}
},
"Entries": {
"/": "baguqeerajks2rqrd57cedyxaoeau7r7mrdnt5qmo7tly5yghp2npwblj3sja"
},
"IsRm": false,
"Metadata": {
"/": {
"bytes": "gBI"
}
},
"PreviousID": {
"/": "baguqeerafcg5fsqu3ei4uy6rrepfth56npmnteuryxixbo5sostozg2o22aa"
},
"Provider": "QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem",
"Signature": {
"/": {
"bytes": "CqsCCAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDuuLoNO5mhRCrLIeKE/zVBX3r3Y9SLbalOtwbYvvh/ounJYq+xatHHCXCx/3OnSDsMfHQxSuDWN8o1YleYPGaoZoLyMy5BvUOMgrRFFbTsiQIbPHo2p+wpRPXSq4wofudz7wBRUdjAPsYkMgUxi/757Z1dpjq+D9FpzwPHqx9nYBGzsKBaqNAcQjb3EbYFrU/xYrl+bYqNCYXgAhXfTtGCKnZAjOiuRHMl2cIy39X50kMfFnXm/gMiVJVf0pxGUsLSfl8HHc0hYLQdYgbSOYBQLamTvzYQJHmfDjgZfmHuuGv+Emh3a/Tq8PNDIgZqf5sAELXQAwIt+Fl/hJK+QgHHAgMBAAESGy9pbmRleGVyL2luZ2VzdC9hZFNpZ25hdHVyZRoiEiAcn2knDT4D30v0e0m65ATwk2ZNfVnaPx6hJkf44V6yaCqAArszYLsDj1YsJc7TavpKFMfzL14zNZGoF1RA2oFHWMIQ9IhkazUTVpoJD1onhFWHeW0eok2j/DUlgg+JZEXJs7q5eoLmH+nZ3knAQAlhC1t04Ujqg28rhqKHUbhm5PIwdNCt+lKeDxmezdfU+2rNB2CpMv8zXCFBhr7iUdBI911zXyOC5jR3aojVN23IWYKMlD027+rENH0V1z6n86+oHhAa92F6Jp5v6kk8CcxJDCQRwKoKM+PGFXgbeV8iuWH0pvmBi0lmyq1oAjBR72pk0Fe3oiu/YN2+QOr8DPgljPpG9++EKYe4tEL49RQFjwLsX/F2quLZ77a7JlZ/J+cs5+w"
}
}
} and the entries file {
"Entries": [
{
"/": {
"bytes": "EiCAqhv8l/LACHU4bqm0egQhgMphMw+l3BOFtUxfFhGaTw=="
}
}
]
} |
I've posted an {
"head": {
"/": "baguqeeraaieajw4fu7jexdwhzadrke5tsdmod3zgh4vujpuyrjtq4kukzppa"
},
"pubkey": {
"/": {
"bytes": "CAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDuuLoNO5mhRCrLIeKE/zVBX3r3Y9SLbalOtwbYvvh/ounJYq+xatHHCXCx/3OnSDsMfHQxSuDWN8o1YleYPGaoZoLyMy5BvUOMgrRFFbTsiQIbPHo2p+wpRPXSq4wofudz7wBRUdjAPsYkMgUxi/757Z1dpjq+D9FpzwPHqx9nYBGzsKBaqNAcQjb3EbYFrU/xYrl+bYqNCYXgAhXfTtGCKnZAjOiuRHMl2cIy39X50kMfFnXm/gMiVJVf0pxGUsLSfl8HHc0hYLQdYgbSOYBQLamTvzYQJHmfDjgZfmHuuGv+Emh3a/Tq8PNDIgZqf5sAELXQAwIt+Fl/hJK+QgHHAgMBAAE"
}
},
"sig": {
"/": {
"bytes": "wVlDejB7yjAEflRS8+lSWrs93ajakVcIS0/wFIbFR7eic5hk99mxcibKTm29oLHOOu1CHkXcgmsdW8mJT0LePYIPZErx+Qv5CV/I94R/1tsJ17+qvH/Wzzd2WPiONu45LkqEr1TICfHS6Ndh2R0VxrNJpiRDGHe4Vgqic6LGW+t1zq0uj7IfQvXc/9mCmaA7VVQ5vjDiGHccX0VMg+UmwNdMaG8vKCn0pa5IYN7cngr8PvLEp05XTRyDrf4HfmowZngR1I89ruc43wluyLQ1wCWqakDfy8nyg0zGwAwva/NfOi9aYGq6h4NslEMaUlAeqcSdU/8vrXBMJ2mDcxW9SA"
}
}
} the head advertisement at baguqeeraaieajw4fu7jexdwhzadrke5tsdmod3zgh4vujpuyrjtq4kukzppa is {
"Addresses": [
"/dns4/elastic-dev.dag.house/tcp/443/wss"
],
"ContextID": {
"/": {
"bytes": ""
}
},
"Entries": {
"/": "bafkreehdwdcefgh4dqkjv67uzcmw7oje"
},
"ExtendedProvider": {
"Override": false,
"Providers": [
{
"Addresses": [
"/dns4/elastic-dev.dag.house/tcp/443/wss"
],
"ID": "QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem",
"Metadata": {
"/": {
"bytes": "gBI"
}
},
"Signature": {
"/": {
"bytes": "CqsCCAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDuuLoNO5mhRCrLIeKE/zVBX3r3Y9SLbalOtwbYvvh/ounJYq+xatHHCXCx/3OnSDsMfHQxSuDWN8o1YleYPGaoZoLyMy5BvUOMgrRFFbTsiQIbPHo2p+wpRPXSq4wofudz7wBRUdjAPsYkMgUxi/757Z1dpjq+D9FpzwPHqx9nYBGzsKBaqNAcQjb3EbYFrU/xYrl+bYqNCYXgAhXfTtGCKnZAjOiuRHMl2cIy39X50kMfFnXm/gMiVJVf0pxGUsLSfl8HHc0hYLQdYgbSOYBQLamTvzYQJHmfDjgZfmHuuGv+Emh3a/Tq8PNDIgZqf5sAELXQAwIt+Fl/hJK+QgHHAgMBAAESKS9pbmRleGVyL2luZ2VzdC9leHRlbmRlZFByb3ZpZGVyU2lnbmF0dXJlGiISIKIWH9hxiGgjAa7yILGiLDfrU9hgPrD0MiHR+crO2ywdKoACf5QvcnRXs7AKQg7DCuHfKp+VJ7woLVA4kpyu5afwHa729kPGGBfGgZ5tD/3nXTskpr+mPrZEyIt5gRLMEaWB2tG9I8zJA7How6d1q7XIA0BI/NcnhlBhocqSjTv/aeyVk9OMCYqXyerJwjeHdyXyMu9YOEHTjvMgui8z2lHCT+Gcx9ZxGjoA5cVHQwzrwTgb/9VqEckiKYLNsOqGRn+Ag69w6l8Kl6j6MWQGuPT0eHBe0Si6jIpOoC0ngIwadYTfxZtRxaV37dbc/ZvfeSnsiN0GxRDxUWmMJklqKJe3Tnm4WMmuMqZD1jLLuJV7/Hn5wu3aQqgbmoRFV8lt2BsBOg"
}
}
},
{
"Addresses": [
"/dns4/freeway-staging.dag.haus/tcp/443/https"
],
"ID": "QmcbxXyuJsVCGyYSKfWJHWXbuUH7meVM3UtEwvwxDtBmX6",
"Metadata": {
"/": {
"bytes": "gID0AQ"
}
},
"Signature": {
"/": {
"bytes": "CqsCCAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQCo2qsb0s6rA3XvDC1/f/vAUV/cmktKMlTUC5ouxi2HQxqyxwJKp/DvzZ4duaMtbRvynVTJSDm6iqoyzguo9+OVDFrH+7i9kituM4SrAoZyDxP14qSZIq0XHjg6FvcJjAKs2IIFJ/fQRBU9Nx8Wd3Nhzd4dk33VgooscPZCRm7XAQwLvZX/eRAXn8LZPisOl23jhFaxE/BTMbNTjK08ry1lMdN6Fug54KDNVk/XAHfp+GlXGTX6lPt3qUhuWyds3jB6H47E9fHAgOHdXOvWHOBatkNiKSCp2Sb4dwZ2/VbldeVklDBfhZZ8ZyEWuLbIOiglJWla1s/2Fv5auC59TnM9AgMBAAESKS9pbmRleGVyL2luZ2VzdC9leHRlbmRlZFByb3ZpZGVyU2lnbmF0dXJlGiISIGVRGIvY/NQukI1IRiJICPBbVhHNcv4KtZavOF3Acb5ZKoACEBTEs7HjP2i0h769+Vr9KWX80VSklpWKszy3NDchOvHm1fKCPTffPgojnwdaDmc7xFhImqMxDMGUvWFsn4GWyLBrhstdi97d5/XFgOAhEKLYsHLyWQxltz32yBrIHkM+jSG/K7r23rTG1ogvZG1v1I9xgKj/JrIsMWVe3oERKz4QQfeMAozV0lRiRnBQNqvPLDB7u47mtSzlvpOeaUAqpRmmwdK0BclJjU88SCfZt/08eGckaPNDfuTFkwWIJRvnP0NQ0AqRY3nLtyh3NO990YwE1/kGIPF/iUI/ay9V09tQSe4XJeFHmg5QJk/w1PKIMesbhqkEnBpzh71WcbpVIQ"
}
}
}
]
},
"IsRm": false,
"Metadata": {
"/": {
"bytes": "gBI"
}
},
"PreviousID": {
"/": "baguqeera7izo6lfcvowkpqg2jjuo2hgglxy5bfbtwla7i4if6zxuhhe3yu4q"
},
"Provider": "QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem",
"Signature": {
"/": {
"bytes": "CqsCCAASpgIwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDuuLoNO5mhRCrLIeKE/zVBX3r3Y9SLbalOtwbYvvh/ounJYq+xatHHCXCx/3OnSDsMfHQxSuDWN8o1YleYPGaoZoLyMy5BvUOMgrRFFbTsiQIbPHo2p+wpRPXSq4wofudz7wBRUdjAPsYkMgUxi/757Z1dpjq+D9FpzwPHqx9nYBGzsKBaqNAcQjb3EbYFrU/xYrl+bYqNCYXgAhXfTtGCKnZAjOiuRHMl2cIy39X50kMfFnXm/gMiVJVf0pxGUsLSfl8HHc0hYLQdYgbSOYBQLamTvzYQJHmfDjgZfmHuuGv+Emh3a/Tq8PNDIgZqf5sAELXQAwIt+Fl/hJK+QgHHAgMBAAESGy9pbmRleGVyL2luZ2VzdC9hZFNpZ25hdHVyZRoiEiD/tWd5d5+hYU0YiA+mUyoCHq4NgpzlNObHnQwJevYAaiqAAg7xHkXd2Ttjziw2lx7KRkXOr36zvGysfq7IP3sT+uwv7FSrFbaLNxZ5rGyLS6kjXP5VvSOOF+IuGcLchB7sLn4DN2gtHZYUvIAejop3a6xAJF7ZVfDTQSCYYN1Y/mdLefwMaVT9psMnUU9v+UsDtIodaDLAX4o/y6VHofQ7CqBXfejtzlGIV4uQmcHsQjcc7u9prGk3MPOw7lyCUwhZtexOeYtHKS3yJDPv12ecGidHAyDyP+GAVQktxU/peXhnBSNsYq85FcNTlFBLlG1+AjQCVRqvzafp+gjFgSBvXcJDkso+u6ApHft/joM0StmAqLOX4lme5p10Td50M5Rkvmo"
}
}
} |
Running # ingest the /head from the dev adverts bucket
./storetheindex admin sync -p QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem --addr "/dns4/dev-ep-ipfs-advertisement.s3.us-west-2.amazonaws.com/tcp/443/https/p2p/QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem" storetheindex logs
|
Rolling back ./storetheindex admin sync -p QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem --addr "/dns4/dev-ep-ipfs-advertisement.s3.us-west-2.amazonaws.com/tcp/443/https/p2p/QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem" storetheindex logs
|
The issue might be with the way how signatures are composed. The line that it fails one invokes |
This was subtling breaking everything. It returned { '/': 'string cid' } instead of a cid instance. That string doesn't have a bytes field so it doesn't get signed correctly when used as a PreviousID. I need to update the lib to error when not passed a CID. License: MIT Signed-off-by: Oli Evans <[email protected]>
I've added a verify command to @web3-storage/ipni to show that it produces advertisements that can be verified with go-libipni storacha/ipni#11 and it works! That helped me narrow it down to the calling code in this PR, which is untyped so wasn't warning me that That's fixed now, and I'm gonna run the tests again. |
Sweet. ran the test again and it blends! ❯ go run main.go ~/Downloads/baguqeera72jbvbgvhm45l2xefmpluy7twhhso4vruj6deprolw4htefaalya.json
Valid! Signed by QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem and it can be ingested by a locally running storetheindex node: $ ./storetheindex admin sync -p QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem --addr "/dns4/dev-ep-ipfs-advertisement.s3.us-west-2.amazonaws.com/tcp/443/https/p2p/QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem"
|
Glad that I worked for you 🙏 |
I have
|
local query on storetheindex of that original test CID works and shows both bitswap and http endpoints ❯ curl http://localhost:3000/multihash/QmWzuLhMZH2RSrxpEYBtE2dWsVPPJMdYxdEkLzr7pbMHQz -Ss | jq
{
"MultihashResults": [
{
"Multihash": "EiCAqhv8l/LACHU4bqm0egQhgMphMw+l3BOFtUxfFhGaTw==",
"ProviderResults": [
{
"ContextID": "YmFndXFlZXJhamtzMnJxcmQ1N2NlZHl4YW9lYXU3cjdtcmRudDVxbW83dGx5NXlnaHAybnB3YmxqM3NqYQ==",
"Metadata": "gBI=",
"Provider": {
"ID": "QmQPF8BeQF5GW3qmXcZLqGp2HqrFmscXaGA7ef8vN2sEem",
"Addrs": [
"/dns4/elastic-dev.dag.house/tcp/443/wss"
]
}
},
{
"ContextID": "YmFndXFlZXJhamtzMnJxcmQ1N2NlZHl4YW9lYXU3cjdtcmRudDVxbW83dGx5NXlnaHAybnB3YmxqM3NqYQ==",
"Metadata": "gID0AQ==",
"Provider": {
"ID": "QmcbxXyuJsVCGyYSKfWJHWXbuUH7meVM3UtEwvwxDtBmX6",
"Addrs": [
"/dns4/freeway-staging.dag.haus/tcp/443/https"
]
}
}
]
}
]
} |
Created production peerId for http as Created a test local advert for that peerid and verified it signs correctly.
|
The staging publisher is creating valid adverts but failing to update |
I've had to create new peerIDs for |
Merging to get this code deployed to staging |
end-to-end test on staging worked ✨ 🎷 🐩 an extended provider for http is listed ❯ curl "https://dev.cid.contact/providers/Qmc5vg9zuLYvDR1wtYHCaxjBHenfCNautRwCjG3n5v5fbs" -sS | jq {
"AddrInfo": {
"ID": "Qmc5vg9zuLYvDR1wtYHCaxjBHenfCNautRwCjG3n5v5fbs",
"Addrs": [
"/dns4/elastic-staging.dag.house/tcp/443/wss"
]
},
"LastAdvertisement": {
"/": "baguqeeramobzcdq64sypa5x2hgikf5jjkw5lpgmtcld6eiinfjjpjsa6tv2a"
},
"LastAdvertisementTime": "2023-05-19T15:08:03Z",
"Publisher": {
"ID": "Qmc5vg9zuLYvDR1wtYHCaxjBHenfCNautRwCjG3n5v5fbs",
"Addrs": [
"/dns4/staging-ep-ipfs-advertisement.s3.us-west-2.amazonaws.com/tcp/443/https"
]
},
"IndexCount": 6,
"ExtendedProviders": {
"Providers": [
{
"ID": "Qmc5vg9zuLYvDR1wtYHCaxjBHenfCNautRwCjG3n5v5fbs",
"Addrs": [
"/dns4/elastic-staging.dag.house/tcp/443/wss"
]
},
{
"ID": "QmX5D2SHKfJEjMfUwKAJZtgXhFzmYvA66Hcc9L3ExuVDXs",
"Addrs": [
"/dns4/freeway-staging.dag.haus/tcp/443/https"
]
}
],
"Metadatas": [
"gBI=",
"gID0AQ=="
]
},
"FrozenAt": null
} the extended provider appears in query results ❯ curl "https://dev.cid.contact/cid/bafybeigqf6fljfatmqi65y3y3h57uqq5aneim43tq6a2wvznwww63y6ela" -sS | jq {
"MultihashResults": [
{
"Multihash": "EiDQL4q0lBNkEe7jeNn7+kIdA0iGc3OHgatXLbWt7ePEWA==",
"ProviderResults": [
{
"ContextID": "YmFndXFlZXJhbnRrMmpkdmg3ZmtkeWxuZTJ6cmp6ZWljaDZmZHRqc2xxZTRrdnh6cDVrcjNvaWNnZm54cQ==",
"Metadata": "gBI=",
"Provider": {
"ID": "Qmc5vg9zuLYvDR1wtYHCaxjBHenfCNautRwCjG3n5v5fbs",
"Addrs": [
"/dns4/elastic-staging.dag.house/tcp/443/wss"
]
}
},
{
"ContextID": "YmFndXFlZXJhbnRrMmpkdmg3ZmtkeWxuZTJ6cmp6ZWljaDZmZHRqc2xxZTRrdnh6cDVrcjNvaWNnZm54cQ==",
"Metadata": "gID0AQ==",
"Provider": {
"ID": "QmX5D2SHKfJEjMfUwKAJZtgXhFzmYvA66Hcc9L3ExuVDXs",
"Addrs": [
"/dns4/freeway-staging.dag.haus/tcp/443/https"
]
}
}
]
}
]
} |
Add support for announcing special ExtendedProvider advertisements. The goal is to advertise our content as being available over bitswap and http.
This implementation hardcodes support for processing messages with the magic string
ExtendedProviderHTTP
which is a signal to publish the next advert in the chain as having the bitswap & http provider details in anExtendedProvider
block.No context or entries are specified on that message so the indexer should interpret it as a request to announce our extra http provider info for all previous and future entries published on this chain.
The logic is hardcoded as the Advertisement must be signed with the private keys of each of the providers. There is no way to pre-sign the extra provider info as the previous advert CID is part of the signature, which we can't guess ahead of time. It is assumed that we will only use this specific code path once (1) (!) to announce this http endpoint. We could use it to announce another http endpoint by updating the env, and the
peerId-http.json
, but in general this whole repo should be replaced with an sst managed esm implementation before that happens.There is currently no support for removing provider info, so we need to be happy to support our http endpoint domain for some time until ipni/storetheindex#1745 lands
The logic of how to serialise and sign advertisements is extracted out to our ipni module: https://github.com/web3-storage/ipni
NOTE This module needs refactoring and updating to the (dag) haus style. I have not tackled that at part of this PR, but have added js-doc and updated modules where i had to.
License: MIT