Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

added urls tests and corrected outdated urls #78

Merged
merged 5 commits into from
Feb 27, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions building-blocks/autobase.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,8 +254,6 @@ await base.view.update()
console.log(base.view.length)
```

More sophisticated indexing might require multiple appends per input node, or reading from the view during `apply` -- both are perfectly valid. The [multiwriter Hyperbee example](https://github.com/holepunchto/autobase/blob/master/examples/autobee-simple.js) shows how this `apply` pattern can be used to build Hypercore-based indexing data structures using this approach.

#### View Creation

#### **`base.start({ apply, unwrap } = {})`**
Expand Down
2 changes: 1 addition & 1 deletion building-blocks/hypercore.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ Creates a replication stream. We should pipe this to another Hypercore instance.

The `isInitiator` argument is a boolean indicating whether a peer is the initiator of the connection (ie the client) or the passive peer waiting for connections (i.e., the server).

> If a P2P swarm like Hyperswarm is being used, whether a peer is an initiator can be determined by checking if the swarm connection is a client socket or a server socket. In Hyperswarm, a user can check that using the [client property on the peer details object](https://github.com/hyperswarm/hyperswarm#swarmonconnection-socket-details--).
> If a P2P swarm like Hyperswarm is being used, whether a peer is an initiator can be determined by checking if the swarm connection is a client socket or a server socket. In Hyperswarm, a user can check that using the [client property on the peer details object](https://github.com/holepunchto/hyperswarm#swarmonconnection-socket-details).



Expand Down
4 changes: 2 additions & 2 deletions building-blocks/hyperdht.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# HyperDHT

The DHT powering Hyperswarm and built on top of [dht-rpc](https://github.com/mafintosh/dht-rpc). The HyperDHT uses a series of holepunching techniques to ensure connectivity works on most networks and is mainly used to facilitate finding and connecting to peers using end-to-end encrypted Noise streams.
The DHT powering Hyperswarm and built on top of [dht-rpc](https://github.com/holepunchto/dht-rpc). The HyperDHT uses a series of holepunching techniques to ensure connectivity works on most networks and is mainly used to facilitate finding and connecting to peers using end-to-end encrypted Noise streams.

In the HyperDHT, peers are identified by a public key, not by an IP address. A public key can be connected regardless of where the peers are located, even if they move between different networks.

Expand Down Expand Up @@ -68,7 +68,7 @@ Create a new DHT node.
| **`bootstrap`** | overwrite the default bootstrap servers, just need to be an array of any known DHT node(s) | Array | `['node1.hyperdht.org:49737', 'node2.hyperdht.org:49737', 'node3.hyperdht.org:49737']` |
| **`keyPair`** | optionally pass the public key and secret key as a key pair to use for server.listen and connect | Object | `null` |

See [dht-rpc](https://github.com/mafintosh/dht-rpc) for more options as HyperDHT inherits from that.
See [dht-rpc](https://github.com/holepunchto/dht-rpc) for more options as HyperDHT inherits from that.

> ℹ️ The default bootstrap servers are publicly served on behalf of the commons. To run a fully isolated DHT, start one or more DHT nodes with an empty bootstrap array (`new DHT({bootstrap:[]})`) and then use the addresses of those nodes as the `bootstrap` option in all other DHT nodes. At least one persistent node is needed for the network to be completely operational.

Expand Down
6 changes: 3 additions & 3 deletions building-blocks/hyperswarm.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ Hyperswarm offers a simple interface to abstract away the complexities of underl
Notable features include:

* An improved UDP holepunching algorithm that uses arbitrary DHT nodes (optionally selected by the connecting peers) to proxy necessary metadata while being maximally privacy-preserving.
* A custom-built transport protocol, [UDX](https://github.com/hyperswarm/libudx), that takes advantage of the holepunching algorithm to avoid unnecessary overhead (it doesn't include handshaking since holepunching takes care of that, for example). It's blazing fast.
* A custom-built transport protocol, [UDX](https://github.com/holepunchto/libudx), that takes advantage of the holepunching algorithm to avoid unnecessary overhead (it doesn't include handshaking since holepunching takes care of that, for example). It's blazing fast.
* A simplified DHT API that closely resembles NodeJS's `net` module, but using public keys instead of IP addresses.

> [GitHub (Hyperswarm)](https://github.com/hyperswarm/hyperswarm)
> [GitHub (Hyperswarm)](https://github.com/holepunchto/hyperswarm)

* [Hyperswarm](../building-blocks/hyperswarm.md)
* [Create a new instance](hyperswarm.md#installation)
Expand Down Expand Up @@ -225,4 +225,4 @@ Stops discovering peers for the given topic.

> If a topic was previously joined in server mode, `leave` will stop announcing the topic on the DHT.
>
>If a topic was previously joined in client mode, `leave` will stop searching for servers announcing the topic.
>If a topic was previously joined in client mode, `leave` will stop searching for servers announcing the topic.
4 changes: 2 additions & 2 deletions helpers/compact-encoding.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A series of binary encoders/decoders for building small and fast parsers and serializers.

> [GitHub (Compact-Encoding)](https://github.com/compact-encoding/compact-encoding)
> [GitHub (Compact-Encoding)](https://github.com/holepunchto/compact-encoding)

* [Compact-Encoding](compact-encoding.md#installation)
* Methods
Expand Down Expand Up @@ -118,7 +118,7 @@ The following encodings are bundled as they are primitives that can be used to b
| `cenc.int48` | Encodes a fixed size int48 using `cenc.uint48` with ZigZag encoding. |
| `cenc.int56` | Encodes a fixed size int56 using `cenc.uint56` with ZigZag encoding |
| `cenc.int64` | Encodes a fixed size int64 using `cenc.uint64` with ZigZag encoding. |
| `cenc.lexint` | Encodes an int using [lexicographic-integer](https://github.com/substack/lexicographic-integer) encoding so that encoded values are lexicographically sorted in ascending numerical order. |
| `cenc.lexint` | Encodes an int using lexicographic-integer encoding so that encoded values are lexicographically sorted in ascending numerical order. |
| `cenc.float32` | Encodes a fixed size float32. |
| `cenc.float64` | Encodes a fixed size float64. |
| `cenc.buffer` | Encodes a buffer with its length uint prefixed. When decoding an empty buffer, `null` is returned. |
Expand Down
2 changes: 1 addition & 1 deletion helpers/protomux.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Multiplex multiple message-oriented protocols over a stream

>[GitHub (Protomux)](https://github.com/mafintosh/protomux)
>[GitHub (Protomux)](https://github.com/holepunchto/protomux)

* [Installation](protomux.md#installation)
* [Basic usage](protomux.md#basic-usage)
Expand Down
4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"main": "index.html",
"type": "module",
"scripts": {
"test": "standard --fix"
"test": "standard --fix",
"check-urls": "node test/urls.tests.cjs"
},
"pear": {
"gui": {
Expand All @@ -26,6 +27,7 @@
"ws": "^8.16.0"
},
"devDependencies": {
"brittle": "^3.4.0",
"standard": "^17.1.0"
}
}
65 changes: 65 additions & 0 deletions tests/urls.test.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
const { readdir, stat, readFile } = require('fs/promises')
const test = require('brittle')
const path = require('path')
const https = require('https')

test('check that all urls can be reached', async (t) => {
const docs = await readMarkdownFiles()

let urls = await Promise.all(docs.map(async (doc) => {
const content = (await readFile(doc)).toString()
const urlRegex = /(?<url>https?:\/\/[^\s)"'`]+)/gi
return content.match(urlRegex)
}))

urls = urls.flat().filter(u => u !== null).filter(u => !u.startsWith('http://localhost'))

const cache = new Map()
const responses = await Promise.all(urls.map(async url => {
if (cache.get(url)) return cache.get(url)
const result = await checkUrl(url.trim())
cache.set(url, result)
return { result, url }
}))

for (const response of responses) {
t.ok(response.result, `${response.url} should return 200 code`)
}
})

async function readMarkdownFiles (folderPath = path.join(__dirname, '..')) {
let result = []
const files = await readdir(folderPath)

for (const file of files) {
const filePath = path.join(folderPath, file)
const stats = await stat(filePath)

if (!filePath.includes('node_modules') && stats.isDirectory()) {
result = result.concat(await readMarkdownFiles(filePath))
} else if (path.extname(filePath) === '.md') {
result.push(filePath)
}
}

return result
}

function checkUrl (url) {
return new Promise((resolve, reject) => {
try {
https.get(url, (res) => {
if (res.statusCode >= 200 && res.statusCode < 300) {
resolve(true)
} else {
resolve(false)
}
}).on('error', () => {
resolve(false)
})
} catch (err) {
console.log(err)
resolve(false)
}
})
}
2 changes: 1 addition & 1 deletion tools/hyperbeam.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

An end-to-end encrypted pipeline for the Internet, utilizing the [`Hyperswarm`](../building-blocks/hyperswarm.md) and Noise Protocol for secure communications.

> [GitHub (Hyperbeam)](https://github.com/mafintosh/hyperbeam)
> [GitHub (Hyperbeam)](https://github.com/holepunchto/hyperbeam)

* [Installation](hyperbeam.md#installation)
* [Basic usage](hyperbeam.md#usage)
Expand Down
Loading