Skip to content

Commit

Permalink
added urls tests and corrected outdated urls (#78)
Browse files Browse the repository at this point in the history
* added urls tests and corrected outdated urls

* removed outdated autobase example link

* change successful test status from <=300 to < 300

* removed index.test.cjs

* fixed test status code

---------

Co-authored-by: rafapaezbas <[email protected]>
  • Loading branch information
rafapaezbas and rafapaezbas authored Feb 27, 2024
1 parent bd1c1e6 commit 924e1fa
Show file tree
Hide file tree
Showing 9 changed files with 78 additions and 13 deletions.
2 changes: 0 additions & 2 deletions building-blocks/autobase.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,8 +254,6 @@ await base.view.update()
console.log(base.view.length)
```

More sophisticated indexing might require multiple appends per input node, or reading from the view during `apply` -- both are perfectly valid. The [multiwriter Hyperbee example](https://github.com/holepunchto/autobase/blob/master/examples/autobee-simple.js) shows how this `apply` pattern can be used to build Hypercore-based indexing data structures using this approach.

#### View Creation

#### **`base.start({ apply, unwrap } = {})`**
Expand Down
2 changes: 1 addition & 1 deletion building-blocks/hypercore.md
Original file line number Diff line number Diff line change
Expand Up @@ -465,7 +465,7 @@ Creates a replication stream. We should pipe this to another Hypercore instance.

The `isInitiator` argument is a boolean indicating whether a peer is the initiator of the connection (ie the client) or the passive peer waiting for connections (i.e., the server).

> If a P2P swarm like Hyperswarm is being used, whether a peer is an initiator can be determined by checking if the swarm connection is a client socket or a server socket. In Hyperswarm, a user can check that using the [client property on the peer details object](https://github.com/hyperswarm/hyperswarm#swarmonconnection-socket-details--).
> If a P2P swarm like Hyperswarm is being used, whether a peer is an initiator can be determined by checking if the swarm connection is a client socket or a server socket. In Hyperswarm, a user can check that using the [client property on the peer details object](https://github.com/holepunchto/hyperswarm#swarmonconnection-socket-details).


Expand Down
4 changes: 2 additions & 2 deletions building-blocks/hyperdht.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# HyperDHT

The DHT powering Hyperswarm and built on top of [dht-rpc](https://github.com/mafintosh/dht-rpc). The HyperDHT uses a series of holepunching techniques to ensure connectivity works on most networks and is mainly used to facilitate finding and connecting to peers using end-to-end encrypted Noise streams.
The DHT powering Hyperswarm and built on top of [dht-rpc](https://github.com/holepunchto/dht-rpc). The HyperDHT uses a series of holepunching techniques to ensure connectivity works on most networks and is mainly used to facilitate finding and connecting to peers using end-to-end encrypted Noise streams.

In the HyperDHT, peers are identified by a public key, not by an IP address. A public key can be connected regardless of where the peers are located, even if they move between different networks.

Expand Down Expand Up @@ -68,7 +68,7 @@ Create a new DHT node.
| **`bootstrap`** | overwrite the default bootstrap servers, just need to be an array of any known DHT node(s) | Array | `['node1.hyperdht.org:49737', 'node2.hyperdht.org:49737', 'node3.hyperdht.org:49737']` |
| **`keyPair`** | optionally pass the public key and secret key as a key pair to use for server.listen and connect | Object | `null` |

See [dht-rpc](https://github.com/mafintosh/dht-rpc) for more options as HyperDHT inherits from that.
See [dht-rpc](https://github.com/holepunchto/dht-rpc) for more options as HyperDHT inherits from that.

> ℹ️ The default bootstrap servers are publicly served on behalf of the commons. To run a fully isolated DHT, start one or more DHT nodes with an empty bootstrap array (`new DHT({bootstrap:[]})`) and then use the addresses of those nodes as the `bootstrap` option in all other DHT nodes. At least one persistent node is needed for the network to be completely operational.
Expand Down
6 changes: 3 additions & 3 deletions building-blocks/hyperswarm.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,10 @@ Hyperswarm offers a simple interface to abstract away the complexities of underl
Notable features include:

* An improved UDP holepunching algorithm that uses arbitrary DHT nodes (optionally selected by the connecting peers) to proxy necessary metadata while being maximally privacy-preserving.
* A custom-built transport protocol, [UDX](https://github.com/hyperswarm/libudx), that takes advantage of the holepunching algorithm to avoid unnecessary overhead (it doesn't include handshaking since holepunching takes care of that, for example). It's blazing fast.
* A custom-built transport protocol, [UDX](https://github.com/holepunchto/libudx), that takes advantage of the holepunching algorithm to avoid unnecessary overhead (it doesn't include handshaking since holepunching takes care of that, for example). It's blazing fast.
* A simplified DHT API that closely resembles NodeJS's `net` module, but using public keys instead of IP addresses.

> [GitHub (Hyperswarm)](https://github.com/hyperswarm/hyperswarm)
> [GitHub (Hyperswarm)](https://github.com/holepunchto/hyperswarm)
* [Hyperswarm](../building-blocks/hyperswarm.md)
* [Create a new instance](hyperswarm.md#installation)
Expand Down Expand Up @@ -225,4 +225,4 @@ Stops discovering peers for the given topic.

> If a topic was previously joined in server mode, `leave` will stop announcing the topic on the DHT.
>
>If a topic was previously joined in client mode, `leave` will stop searching for servers announcing the topic.
>If a topic was previously joined in client mode, `leave` will stop searching for servers announcing the topic.
4 changes: 2 additions & 2 deletions helpers/compact-encoding.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

A series of binary encoders/decoders for building small and fast parsers and serializers.

> [GitHub (Compact-Encoding)](https://github.com/compact-encoding/compact-encoding)
> [GitHub (Compact-Encoding)](https://github.com/holepunchto/compact-encoding)
* [Compact-Encoding](compact-encoding.md#installation)
* Methods
Expand Down Expand Up @@ -118,7 +118,7 @@ The following encodings are bundled as they are primitives that can be used to b
| `cenc.int48` | Encodes a fixed size int48 using `cenc.uint48` with ZigZag encoding. |
| `cenc.int56` | Encodes a fixed size int56 using `cenc.uint56` with ZigZag encoding |
| `cenc.int64` | Encodes a fixed size int64 using `cenc.uint64` with ZigZag encoding. |
| `cenc.lexint` | Encodes an int using [lexicographic-integer](https://github.com/substack/lexicographic-integer) encoding so that encoded values are lexicographically sorted in ascending numerical order. |
| `cenc.lexint` | Encodes an int using lexicographic-integer encoding so that encoded values are lexicographically sorted in ascending numerical order. |
| `cenc.float32` | Encodes a fixed size float32. |
| `cenc.float64` | Encodes a fixed size float64. |
| `cenc.buffer` | Encodes a buffer with its length uint prefixed. When decoding an empty buffer, `null` is returned. |
Expand Down
2 changes: 1 addition & 1 deletion helpers/protomux.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

Multiplex multiple message-oriented protocols over a stream

>[GitHub (Protomux)](https://github.com/mafintosh/protomux)
>[GitHub (Protomux)](https://github.com/holepunchto/protomux)
* [Installation](protomux.md#installation)
* [Basic usage](protomux.md#basic-usage)
Expand Down
4 changes: 3 additions & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,8 @@
"main": "index.html",
"type": "module",
"scripts": {
"test": "standard --fix"
"test": "standard --fix",
"check-urls": "node test/urls.tests.cjs"
},
"pear": {
"gui": {
Expand All @@ -26,6 +27,7 @@
"ws": "^8.16.0"
},
"devDependencies": {
"brittle": "^3.4.0",
"standard": "^17.1.0"
}
}
65 changes: 65 additions & 0 deletions tests/urls.test.cjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
const { readdir, stat, readFile } = require('fs/promises')
const test = require('brittle')
const path = require('path')
const https = require('https')

test('check that all urls can be reached', async (t) => {
const docs = await readMarkdownFiles()

let urls = await Promise.all(docs.map(async (doc) => {
const content = (await readFile(doc)).toString()
const urlRegex = /(?<url>https?:\/\/[^\s)"'`]+)/gi
return content.match(urlRegex)
}))

urls = urls.flat().filter(u => u !== null).filter(u => !u.startsWith('http://localhost'))

const cache = new Map()
const responses = await Promise.all(urls.map(async url => {
if (cache.get(url)) return cache.get(url)
const result = await checkUrl(url.trim())
cache.set(url, result)
return { result, url }
}))

for (const response of responses) {
t.ok(response.result, `${response.url} should return 200 code`)
}
})

async function readMarkdownFiles (folderPath = path.join(__dirname, '..')) {
let result = []
const files = await readdir(folderPath)

for (const file of files) {
const filePath = path.join(folderPath, file)
const stats = await stat(filePath)

if (!filePath.includes('node_modules') && stats.isDirectory()) {
result = result.concat(await readMarkdownFiles(filePath))
} else if (path.extname(filePath) === '.md') {
result.push(filePath)
}
}

return result
}

function checkUrl (url) {
return new Promise((resolve, reject) => {
try {
https.get(url, (res) => {
if (res.statusCode >= 200 && res.statusCode < 300) {
resolve(true)
} else {
resolve(false)
}
}).on('error', () => {
resolve(false)
})
} catch (err) {
console.log(err)
resolve(false)
}
})
}
2 changes: 1 addition & 1 deletion tools/hyperbeam.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

An end-to-end encrypted pipeline for the Internet, utilizing the [`Hyperswarm`](../building-blocks/hyperswarm.md) and Noise Protocol for secure communications.

> [GitHub (Hyperbeam)](https://github.com/mafintosh/hyperbeam)
> [GitHub (Hyperbeam)](https://github.com/holepunchto/hyperbeam)
* [Installation](hyperbeam.md#installation)
* [Basic usage](hyperbeam.md#usage)
Expand Down

0 comments on commit 924e1fa

Please sign in to comment.