filter local addresses (for WAN) and localhost addresses (for LAN)

added tests for addrfilter fix: don't add unresponsive DHT servers to the Routing Table (libp2p#820) * added check to avoid adding unresponsive dht peers to the dht routing table * removed lock in adding peers to the rt * made variable names more meaningful * fixed network loop and corrected tests * added UsefulPeer() references from current PR * go mod tidy * added delay in TestRefreshBelowMinRTThreshold * addressed review * go mod tidy * addressed Jorropo review * added comments * removed state of peers probed recently * fix conflicts merge * updated deps * added optimizations documentation * Update dht.go * updated md files --------- Co-authored-by: Jorropo <[email protected]> fix: don't add unresponsive DHT servers to the Routing Table (libp2p#820) * added check to avoid adding unresponsive dht peers to the dht routing table * removed lock in adding peers to the rt * made variable names more meaningful * fixed network loop and corrected tests * added UsefulPeer() references from current PR * go mod tidy * added delay in TestRefreshBelowMinRTThreshold * addressed review * go mod tidy * addressed Jorropo review * added comments * removed state of peers probed recently * fix conflicts merge * updated deps * added optimizations documentation * Update dht.go * updated md files --------- Co-authored-by: Jorropo <[email protected]> (cherry picked from commit 8c9fdff) chore: release v0.24.0 chore: Update .github/workflows/stale.yml [skip ci] fix: leaking go routines fix: bump kbucket for abba bug refactor: remove goprocess fix: decrease tests noise, update kbucket and fix fixRTIUfNeeded chore: release v0.24.1 chore: update go-libp2p to a non deadlocky version chore: release v0.24.2 chore: Update .github/workflows/stale.yml [skip ci] tracing: fix DHT keys as string attribute not being valid utf-8 chore: release v0.24.3 fix the server mode log in handleNewMessage by changing it to debug chore: delete templates [skip ci] (libp2p#861) ci: uci/copy-templates (libp2p#862) * chore: add or force update .github/workflows/go-test.yml * chore: add or force update .github/workflows/go-check.yml * chore: add or force update .github/workflows/releaser.yml * chore: add or force update .github/workflows/release-check.yml * chore: add or force update .github/workflows/tagpush.yml chore: bump go.mod to Go 1.20 and run go fix chore: bump go-libp2p and/or quic-go to latest version tracing: add protocol messages client tracing chore: release v0.25.0 add provider record addresses to peerstore fixes issue libp2p#868 fixing base32 import fix: apply addrFilters in the dht (libp2p#872) * fix: correctly apply addrFilters in the dht This still does not do the fullrt client but it wasn't doing it before either. * Add address filter tests * use channel instead of waitgroup to catch timeouts * filter multiaddresses when serving provider records --------- Co-authored-by: Dennis Trautwein <[email protected]> fix: properly iterate in tracing for protocol messenger MB Copy Paste Typo, This caused panics while tracing. chore: use go-libp2p-routing-helpers for tracing needs go-libp2p-routing-helpers has an optimized implementation that does nothing if we are not tracing, it also properly log all IO of the request. perf: don't buffer the output of FindProvidersAsync We always select it against ctx and we don't rely on the fact it will always be non blocking since when count is zero we can't realistically preallocate. Buffering use more memory makes more garbage and is less efficient than direct copies. chore: release v0.25.1 add ctx canceled err check return canceled err not retry reset stream and return err when ctx canceled release v0.25.2 chore: Update .github/workflows/stale.yml [skip ci] chore: bump go.mod to Go 1.21 and run go fix chore: run go mod tidy chore: add or force update .github/workflows/go-test.yml chore: add or force update .github/workflows/go-check.yml chore: add or force update .github/workflows/releaser.yml chore: add or force update .github/workflows/release-check.yml chore: add or force update .github/workflows/tagpush.yml chore: add or force update .github/workflows/go-test.yml chore: add or force update .github/workflows/go-check.yml chore: add or force update .github/workflows/releaser.yml chore: add or force update .github/workflows/release-check.yml chore: add or force update .github/workflows/tagpush.yml chore: add or force update .github/workflows/go-test.yml chore: add or force update .github/workflows/go-check.yml chore: add or force update .github/workflows/releaser.yml chore: add or force update .github/workflows/release-check.yml chore: add or force update .github/workflows/tagpush.yml findnode(self) now returns multiple peers don't lookup check if not enough peers in table enhanced tests moved rt size check after request Upgrade to go-log v2.5.1
cortze · Jul 30, 2024 · 12756f0 · 12756f0
1 parent 8842756
commit 12756f0
Show file tree

Hide file tree

Showing 41 changed files with 1,375 additions and 1,378 deletions.
diff --git a/.github/workflows/automerge.yml b/.github/workflows/automerge.yml
diff --git a/.github/workflows/go-check.yml b/.github/workflows/go-check.yml
@@ -1,67 +1,18 @@
-# File managed by web3-bot. DO NOT EDIT.
-# See https://github.com/protocol/.github/ for details.
-
-on: [push, pull_request]
 name: Go Checks
 
+on:
+  pull_request:
+  push:
+    branches: ["master"]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
+  cancel-in-progress: true
+
 jobs:
-  unit:
-    runs-on: ubuntu-latest
-    name: All
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - id: config
-        uses: protocol/.github/.github/actions/read-config@master
-      - uses: actions/setup-go@v3
-        with:
-          go-version: 1.20.x
-      - name: Run repo-specific setup
-        uses: ./.github/actions/go-check-setup
-        if: hashFiles('./.github/actions/go-check-setup') != ''
-      - name: Install staticcheck
-        run: go install honnef.co/go/tools/cmd/staticcheck@4970552d932f48b71485287748246cf3237cebdf # 2023.1 (v0.4.0)
-      - name: Check that go.mod is tidy
-        uses: protocol/[email protected]
-        with:
-          run: |
-            go mod tidy
-            if [[ -n $(git ls-files --other --exclude-standard --directory -- go.sum) ]]; then
-              echo "go.sum was added by go mod tidy"
-              exit 1
-            fi
-            git diff --exit-code -- go.sum go.mod
-      - name: gofmt
-        if: success() || failure() # run this step even if the previous one failed
-        run: |
-          out=$(gofmt -s -l .)
-          if [[ -n "$out" ]]; then
-            echo $out | awk '{print "::error file=" $0 ",line=0,col=0::File is not gofmt-ed."}'
-            exit 1
-          fi
-      - name: go vet
-        if: success() || failure() # run this step even if the previous one failed
-        uses: protocol/[email protected]
-        with:
-          run: go vet ./...
-      - name: staticcheck
-        if: success() || failure() # run this step even if the previous one failed
-        uses: protocol/[email protected]
-        with:
-          run: |
-            set -o pipefail
-            staticcheck ./... | sed -e 's@\(.*\)\.go@./\1.go@g'
-      - name: go generate
-        uses: protocol/[email protected]
-        if: (success() || failure()) && fromJSON(steps.config.outputs.json).gogenerate == true
-        with:
-          run: |
-            git clean -fd # make sure there aren't untracked files / directories
-            go generate -x ./...
-            # check if go generate modified or added any files
-            if ! $(git add . && git diff-index HEAD --exit-code --quiet); then
-              echo "go generated caused changes to the repository:"
-              git status --short
-              exit 1
-            fi
+  go-check:
+    uses: ipdxco/unified-github-workflows/.github/workflows/[email protected]
diff --git a/.github/workflows/go-test.yml b/.github/workflows/go-test.yml
@@ -1,76 +1,20 @@
-# File managed by web3-bot. DO NOT EDIT.
-# See https://github.com/protocol/.github/ for details.
-
-on: [push, pull_request]
 name: Go Test
 
+on:
+  pull_request:
+  push:
+    branches: ["master"]
+  workflow_dispatch:
+
+permissions:
+  contents: read
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.event_name }}-${{ github.event_name == 'push' && github.sha || github.ref }}
+  cancel-in-progress: true
+
 jobs:
-  unit:
-    strategy:
-      fail-fast: false
-      matrix:
-        os: [ "ubuntu", "windows", "macos" ]
-        go: ["1.19.x","1.20.x"]
-    env:
-      COVERAGES: ""
-    runs-on: ${{ fromJSON(vars[format('UCI_GO_TEST_RUNNER_{0}', matrix.os)] || format('"{0}-latest"', matrix.os)) }}
-    name: ${{ matrix.os }} (go ${{ matrix.go }})
-    steps:
-      - uses: actions/checkout@v3
-        with:
-          submodules: recursive
-      - id: config
-        uses: protocol/.github/.github/actions/read-config@master
-      - uses: actions/setup-go@v3
-        with:
-          go-version: ${{ matrix.go }}
-      - name: Go information
-        run: |
-          go version
-          go env
-      - name: Use msys2 on windows
-        if: matrix.os == 'windows'
-        shell: bash
-        # The executable for msys2 is also called bash.cmd
-        #   https://github.com/actions/virtual-environments/blob/main/images/win/Windows2019-Readme.md#shells
-        # If we prepend its location to the PATH
-        #   subsequent 'shell: bash' steps will use msys2 instead of gitbash
-        run: echo "C:/msys64/usr/bin" >> $GITHUB_PATH
-      - name: Run repo-specific setup
-        uses: ./.github/actions/go-test-setup
-        if: hashFiles('./.github/actions/go-test-setup') != ''
-      - name: Run tests
-        if: contains(fromJSON(steps.config.outputs.json).skipOSes, matrix.os) == false
-        uses: protocol/[email protected]
-        with:
-          # Use -coverpkg=./..., so that we include cross-package coverage.
-          # If package ./A imports ./B, and ./A's tests also cover ./B,
-          # this means ./B's coverage will be significantly higher than 0%.
-          run: go test -v -shuffle=on -coverprofile=module-coverage.txt -coverpkg=./... ./...
-      - name: Run tests (32 bit)
-        # can't run 32 bit tests on OSX.
-        if: matrix.os != 'macos' &&
-          fromJSON(steps.config.outputs.json).skip32bit != true &&
-          contains(fromJSON(steps.config.outputs.json).skipOSes, matrix.os) == false
-        uses: protocol/[email protected]
-        env:
-          GOARCH: 386
-        with:
-          run: |
-            export "PATH=$PATH_386:$PATH"
-            go test -v -shuffle=on ./...
-      - name: Run tests with race detector
-        # speed things up. Windows and OSX VMs are slow
-        if: matrix.os == 'ubuntu' &&
-          contains(fromJSON(steps.config.outputs.json).skipOSes, matrix.os) == false
-        uses: protocol/[email protected]
-        with:
-          run: go test -v -race ./...
-      - name: Collect coverage files
-        shell: bash
-        run: echo "COVERAGES=$(find . -type f -name 'module-coverage.txt' | tr -s '\n' ',' | sed 's/,$//')" >> $GITHUB_ENV
-      - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@d9f34f8cd5cb3b3eb79b3e4b5dae3a16df499a70 # v3.1.1
-        with:
-          files: '${{ env.COVERAGES }}'
-          env_vars: OS=${{ matrix.os }}, GO=${{ matrix.go }}
+  go-test:
+    uses: ipdxco/unified-github-workflows/.github/workflows/[email protected]
+    secrets:
+      CODECOV_TOKEN: ${{ secrets.CODECOV_TOKEN }}
diff --git a/.github/workflows/release-check.yml b/.github/workflows/release-check.yml
@@ -1,13 +1,19 @@
-# File managed by web3-bot. DO NOT EDIT.
-# See https://github.com/protocol/.github/ for details.
-
 name: Release Checker
+
 on:
   pull_request_target:
     paths: [ 'version.json' ]
+    types: [ opened, synchronize, reopened, labeled, unlabeled ]
+  workflow_dispatch:
+
+permissions:
+  contents: write
+  pull-requests: write
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
 
 jobs:
   release-check:
-    uses: protocol/.github/.github/workflows/release-check.yml@master
-    with:
-      go-version: 1.20.x
+    uses: ipdxco/unified-github-workflows/.github/workflows/[email protected]
diff --git a/.github/workflows/releaser.yml b/.github/workflows/releaser.yml
@@ -1,11 +1,17 @@
-# File managed by web3-bot. DO NOT EDIT.
-# See https://github.com/protocol/.github/ for details.
-
 name: Releaser
+
 on:
   push:
     paths: [ 'version.json' ]
+  workflow_dispatch:
+
+permissions:
+  contents: write
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.sha }}
+  cancel-in-progress: true
 
 jobs:
   releaser:
-    uses: protocol/.github/.github/workflows/releaser.yml@master
+    uses: ipdxco/unified-github-workflows/.github/workflows/releaser.yml@v1.0
diff --git a/.github/workflows/stale.yml b/.github/workflows/stale.yml
@@ -2,25 +2,12 @@ name: Close and mark stale issue
 
 on:
   schedule:
-  - cron: '0 0 * * *'
+    - cron: '0 0 * * *'
+
+permissions:
+  issues: write
+  pull-requests: write
 
 jobs:
   stale:
-
-    runs-on: ubuntu-latest
-    permissions:
-      issues: write
-      pull-requests: write
-
-    steps:
-    - uses: actions/stale@v3
-      with:
-        repo-token: ${{ secrets.GITHUB_TOKEN }}
-        stale-issue-message: 'Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.'
-        close-issue-message: 'This issue was closed because it is missing author input.'
-        stale-issue-label: 'kind/stale'
-        any-of-labels: 'need/author-input'
-        exempt-issue-labels: 'need/triage,need/community-input,need/maintainer-input,need/maintainers-input,need/analysis,status/blocked,status/in-progress,status/ready,status/deferred,status/inactive'
-        days-before-issue-stale: 6
-        days-before-issue-close: 7
-        enable-statistics: true
+    uses: pl-strflt/.github/.github/workflows/[email protected]
diff --git a/.github/workflows/tagpush.yml b/.github/workflows/tagpush.yml
@@ -1,12 +1,18 @@
-# File managed by web3-bot. DO NOT EDIT.
-# See https://github.com/protocol/.github/ for details.
-
 name: Tag Push Checker
+
 on:
   push:
     tags:
       - v*
 
+permissions:
+  contents: read
+  issues: write
+
+concurrency:
+  group: ${{ github.workflow }}-${{ github.ref }}
+  cancel-in-progress: true
+
 jobs:
   releaser:
-    uses: protocol/.github/.github/workflows/tagpush.yml@master
+    uses: ipdxco/unified-github-workflows/.github/workflows/tagpush.yml@v1.0
diff --git a/README.md b/README.md
@@ -11,6 +11,7 @@
 
 - [Install](#install)
 - [Usage](#usage)
+- [Optimizations](#optimizations)
 - [Contribute](#contribute)
 - [Maintainers](#maintainers)
 - [License](#license)
@@ -21,6 +22,10 @@
 go get github.com/libp2p/go-libp2p-kad-dht
 ```
 
+## Optimizations
+
+Client-side optimizations are described in [optimizations.md](./optimizations.md)
+
 ## Usage
 
 Go to https://godoc.org/github.com/libp2p/go-libp2p-kad-dht.

diff --git a/crawler/crawler.go b/crawler/crawler.go
@@ -10,7 +10,7 @@ import (
 	"github.com/libp2p/go-libp2p/core/peer"
 	"github.com/libp2p/go-libp2p/core/protocol"
 
-	logging "github.com/ipfs/go-log"
+	logging "github.com/ipfs/go-log/v2"
 	//lint:ignore SA1019 TODO migrate away from gogo pb
 	"github.com/libp2p/go-msgio/protoio"