Skip to content

Commit

Permalink
Merging Swarm 2.0 master (#32)
Browse files Browse the repository at this point in the history
* fix(stamper): global lock stamper across multiple upload sessions (ethersphere#4578)
* fix: strategy and fetch timeout parsing (ethersphere#4579)
* feat: neighborhood suggester config (ethersphere#4580)
* feat: add codeql.yml (ethersphere#4334)
* feat: add reserveSizeWithinRadius to status protocol (ethersphere#4585)
* fix: missing 200 response (ethersphere#4526)
* feat: pinned reference integrity check API (ethersphere#4573)
* fix(redundancy/getter): wait for recovery and return error (ethersphere#4581)
* fix(pushsync): store the chunk locally when no peers are available fo… (ethersphere#4597)
* fix(redundancy): on by default when downloading (ethersphere#4602)
* fix: add missing openapi spec (ethersphere#4598)
* feat: bzz resource info API (ethersphere#4588)
* fix(redundancy): bzz unit test (ethersphere#4603)
* feat: redundancy ci (ethersphere#4591)
* chore: bump github.com/quic-go/quic-go from 0.38.1 to 0.38.2 (ethersphere#4534)
* feat: split input file to chunks with specified redundancy (ethersphere#4600)
* perf(getter): cancel inflight requests if enough chunks are fetched for recovery (ethersphere#4608)
* fix: store dir error info (ethersphere#4605)
* chore: remove repetitive words (ethersphere#4611)
* fix: use neighborhood suggester only on mainnet (ethersphere#4612)
* feat: alternative withdrawal address (ethersphere#4606)
* fix(seg65) (ethersphere#4604)
* fix(getter): redundancy getter cleanup (ethersphere#4610)
* feat: v2 (ethersphere#4615)
* fix(pin_integrity): changed route and added openapi (ethersphere#4616)
* fix: missing v2 in the makefile and goreleaser (ethersphere#4622)
  • Loading branch information
aranyia authored Apr 3, 2024
1 parent 507716d commit f37a2c2
Show file tree
Hide file tree
Showing 587 changed files with 4,474 additions and 3,083 deletions.
6 changes: 6 additions & 0 deletions .github/workflows/beekeeper.yml
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,12 @@ jobs:
- name: Test staking
id: stake
run: timeout ${TIMEOUT} beekeeper check --cluster-name local-dns --checks ci-stake
- name: Test withdraw
id: withdraw
run: timeout ${TIMEOUT} bash -c 'until beekeeper check --cluster-name local-dns --checks ci-withdraw; do echo "waiting for withdraw..."; sleep .3; done'
- name: Test redundancy
id: redundancy
run: timeout ${TIMEOUT} beekeeper check --cluster-name local-dns --checks ci-redundancy
- name: Collect debug artifacts
if: failure()
run: |
Expand Down
76 changes: 76 additions & 0 deletions .github/workflows/codeql.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"
permissions: read-all
on:
push:
branches: [ "master" ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ "master" ]
schedule:
- cron: '23 1 * * 2'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Use only 'java' to analyze code written in Java, Kotlin or both
# Use only 'javascript' to analyze code written in JavaScript, TypeScript or both
# Learn more about CodeQL language support at https://aka.ms/codeql-docs/language-support

steps:
- name: Checkout repository
uses: actions/checkout@v3

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v2
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.

# Details on CodeQL's query packs refer to : https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality


# Autobuild attempts to build any compiled languages (C/C++, C#, Go, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v2

# ℹ️ Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun

# If the Autobuild fails above, remove it and uncomment the following three lines.
# modify them (or add more) to build your code if your project, please refer to the EXAMPLE below for guidance.

# - run: |
# echo "Run, Build Application using script"
# ./location_of_script_within_repo/buildscript.sh

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v2
with:
category: "/language:${{matrix.language}}"
50 changes: 25 additions & 25 deletions .goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,11 @@ builds:
- -trimpath
ldflags:
- -s -w
- -X github.com/ethersphere/bee.version={{ .Version }}
- -X github.com/ethersphere/bee.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
- -X github.com/ethersphere/bee/v2.version={{ .Version }}
- -X github.com/ethersphere/bee/v2.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee/v2.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/v2/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/v2/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
env:
- CGO_ENABLED=0
goos:
Expand All @@ -39,11 +39,11 @@ builds:
- -trimpath
ldflags:
- -s -w
- -X github.com/ethersphere/bee.version={{ .Version }}
- -X github.com/ethersphere/bee.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
- -X github.com/ethersphere/bee/v2.version={{ .Version }}
- -X github.com/ethersphere/bee/v2.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee/v2.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/v2/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/v2/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
env:
- CGO_ENABLED=0
goos:
Expand All @@ -65,11 +65,11 @@ builds:
- -trimpath
ldflags:
- -s -w
- -X github.com/ethersphere/bee.version={{ .Version }}
- -X github.com/ethersphere/bee.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
- -X github.com/ethersphere/bee/v2.version={{ .Version }}
- -X github.com/ethersphere/bee/v2.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee/v2.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/v2/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/v2/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
env:
- CGO_ENABLED=0
goos:
Expand All @@ -85,11 +85,11 @@ builds:
- -trimpath
ldflags:
- -s -w
- -X github.com/ethersphere/bee.version={{ .Version }}
- -X github.com/ethersphere/bee.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
- -X github.com/ethersphere/bee/v2.version={{ .Version }}
- -X github.com/ethersphere/bee/v2.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee/v2.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/v2/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/v2/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
env:
- CGO_ENABLED=0
goos:
Expand All @@ -104,11 +104,11 @@ builds:
- -trimpath
ldflags:
- -s -w
- -X github.com/ethersphere/bee.version={{ .Version }}
- -X github.com/ethersphere/bee.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
- -X github.com/ethersphere/bee/v2.version={{ .Version }}
- -X github.com/ethersphere/bee/v2.commitHash={{ .ShortCommit }}
- -X github.com/ethersphere/bee/v2.commitTime={{ .CommitTimestamp }}
- -X github.com/ethersphere/bee/v2/pkg/api.Version={{ .Env.BEE_API_VERSION }}
- -X github.com/ethersphere/bee/v2/pkg/debugapi.Version={{ .Env.BEE_DEBUG_API_VERSION }}
env:
- CGO_ENABLED=0
goos:
Expand Down
14 changes: 7 additions & 7 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,13 @@ COMMIT_HASH ?= "$(shell git describe --long --dirty --always --match "" || true)
CLEAN_COMMIT ?= "$(shell git describe --long --always --match "" || true)"
COMMIT_TIME ?= "$(shell git show -s --format=%ct $(CLEAN_COMMIT) || true)"
LDFLAGS ?= -s -w \
-X github.com/ethersphere/bee.version="$(VERSION)" \
-X github.com/ethersphere/bee.commitHash="$(COMMIT_HASH)" \
-X github.com/ethersphere/bee.commitTime="$(COMMIT_TIME)" \
-X github.com/ethersphere/bee/pkg/api.Version="$(BEE_API_VERSION)" \
-X github.com/ethersphere/bee/pkg/api.DebugVersion="$(BEE_DEBUG_API_VERSION)" \
-X github.com/ethersphere/bee/pkg/p2p/libp2p.reachabilityOverridePublic="$(REACHABILITY_OVERRIDE_PUBLIC)" \
-X github.com/ethersphere/bee/pkg/postage/listener.batchFactorOverridePublic="$(BATCHFACTOR_OVERRIDE_PUBLIC)"
-X github.com/ethersphere/bee/v2.version="$(VERSION)" \
-X github.com/ethersphere/bee/v2.commitHash="$(COMMIT_HASH)" \
-X github.com/ethersphere/bee/v2.commitTime="$(COMMIT_TIME)" \
-X github.com/ethersphere/bee/v2/pkg/api.Version="$(BEE_API_VERSION)" \
-X github.com/ethersphere/bee/v2/pkg/api.DebugVersion="$(BEE_DEBUG_API_VERSION)" \
-X github.com/ethersphere/bee/v2/pkg/p2p/libp2p.reachabilityOverridePublic="$(REACHABILITY_OVERRIDE_PUBLIC)" \
-X github.com/ethersphere/bee/v2/pkg/postage/listener.batchFactorOverridePublic="$(BATCHFACTOR_OVERRIDE_PUBLIC)"

.PHONY: all
all: build lint test-race binary
Expand Down
138 changes: 71 additions & 67 deletions cmd/bee/cmd/cmd.go
Original file line number Diff line number Diff line change
Expand Up @@ -13,78 +13,80 @@ import (
"strings"
"time"

chaincfg "github.com/ethersphere/bee/pkg/config"
"github.com/ethersphere/bee/pkg/log"
"github.com/ethersphere/bee/pkg/node"
"github.com/ethersphere/bee/pkg/swarm"
chaincfg "github.com/ethersphere/bee/v2/pkg/config"
"github.com/ethersphere/bee/v2/pkg/log"
"github.com/ethersphere/bee/v2/pkg/node"
"github.com/ethersphere/bee/v2/pkg/swarm"
"github.com/spf13/cobra"
"github.com/spf13/viper"
)

const (
optionNameDataDir = "data-dir"
optionNameCacheCapacity = "cache-capacity"
optionNameDBOpenFilesLimit = "db-open-files-limit"
optionNameDBBlockCacheCapacity = "db-block-cache-capacity"
optionNameDBWriteBufferSize = "db-write-buffer-size"
optionNameDBDisableSeeksCompaction = "db-disable-seeks-compaction"
optionNamePassword = "password"
optionNamePasswordFile = "password-file"
optionNameAPIAddr = "api-addr"
optionNameP2PAddr = "p2p-addr"
optionNameNATAddr = "nat-addr"
optionNameP2PWSEnable = "p2p-ws-enable"
optionNameDebugAPIEnable = "debug-api-enable"
optionNameDebugAPIAddr = "debug-api-addr"
optionNameBootnodes = "bootnode"
optionNameNetworkID = "network-id"
optionWelcomeMessage = "welcome-message"
optionCORSAllowedOrigins = "cors-allowed-origins"
optionNameTracingEnabled = "tracing-enable"
optionNameTracingEndpoint = "tracing-endpoint"
optionNameTracingHost = "tracing-host"
optionNameTracingPort = "tracing-port"
optionNameTracingServiceName = "tracing-service-name"
optionNameVerbosity = "verbosity"
optionNamePaymentThreshold = "payment-threshold"
optionNamePaymentTolerance = "payment-tolerance-percent"
optionNamePaymentEarly = "payment-early-percent"
optionNameResolverEndpoints = "resolver-options"
optionNameBootnodeMode = "bootnode-mode"
optionNameClefSignerEnable = "clef-signer-enable"
optionNameClefSignerEndpoint = "clef-signer-endpoint"
optionNameClefSignerEthereumAddress = "clef-signer-ethereum-address"
optionNameSwapEndpoint = "swap-endpoint" // deprecated: use rpc endpoint instead
optionNameBlockchainRpcEndpoint = "blockchain-rpc-endpoint"
optionNameSwapFactoryAddress = "swap-factory-address"
optionNameSwapInitialDeposit = "swap-initial-deposit"
optionNameSwapEnable = "swap-enable"
optionNameChequebookEnable = "chequebook-enable"
optionNameSwapDeploymentGasPrice = "swap-deployment-gas-price"
optionNameFullNode = "full-node"
optionNamePostageContractAddress = "postage-stamp-address"
optionNamePostageContractStartBlock = "postage-stamp-start-block"
optionNamePriceOracleAddress = "price-oracle-address"
optionNameRedistributionAddress = "redistribution-address"
optionNameStakingAddress = "staking-address"
optionNameBlockTime = "block-time"
optionWarmUpTime = "warmup-time"
optionNameMainNet = "mainnet"
optionNameRetrievalCaching = "cache-retrieval"
optionNameDevReserveCapacity = "dev-reserve-capacity"
optionNameResync = "resync"
optionNamePProfBlock = "pprof-profile"
optionNamePProfMutex = "pprof-mutex"
optionNameStaticNodes = "static-nodes"
optionNameAllowPrivateCIDRs = "allow-private-cidrs"
optionNameSleepAfter = "sleep-after"
optionNameRestrictedAPI = "restricted"
optionNameTokenEncryptionKey = "token-encryption-key"
optionNameAdminPasswordHash = "admin-password"
optionNameUsePostageSnapshot = "use-postage-snapshot"
optionNameStorageIncentivesEnable = "storage-incentives-enable"
optionNameStateStoreCacheCapacity = "statestore-cache-capacity"
optionNameTargetNeighborhood = "target-neighborhood"
optionNameDataDir = "data-dir"
optionNameCacheCapacity = "cache-capacity"
optionNameDBOpenFilesLimit = "db-open-files-limit"
optionNameDBBlockCacheCapacity = "db-block-cache-capacity"
optionNameDBWriteBufferSize = "db-write-buffer-size"
optionNameDBDisableSeeksCompaction = "db-disable-seeks-compaction"
optionNamePassword = "password"
optionNamePasswordFile = "password-file"
optionNameAPIAddr = "api-addr"
optionNameP2PAddr = "p2p-addr"
optionNameNATAddr = "nat-addr"
optionNameP2PWSEnable = "p2p-ws-enable"
optionNameDebugAPIEnable = "debug-api-enable"
optionNameDebugAPIAddr = "debug-api-addr"
optionNameBootnodes = "bootnode"
optionNameNetworkID = "network-id"
optionWelcomeMessage = "welcome-message"
optionCORSAllowedOrigins = "cors-allowed-origins"
optionNameTracingEnabled = "tracing-enable"
optionNameTracingEndpoint = "tracing-endpoint"
optionNameTracingHost = "tracing-host"
optionNameTracingPort = "tracing-port"
optionNameTracingServiceName = "tracing-service-name"
optionNameVerbosity = "verbosity"
optionNamePaymentThreshold = "payment-threshold"
optionNamePaymentTolerance = "payment-tolerance-percent"
optionNamePaymentEarly = "payment-early-percent"
optionNameResolverEndpoints = "resolver-options"
optionNameBootnodeMode = "bootnode-mode"
optionNameClefSignerEnable = "clef-signer-enable"
optionNameClefSignerEndpoint = "clef-signer-endpoint"
optionNameClefSignerEthereumAddress = "clef-signer-ethereum-address"
optionNameSwapEndpoint = "swap-endpoint" // deprecated: use rpc endpoint instead
optionNameBlockchainRpcEndpoint = "blockchain-rpc-endpoint"
optionNameSwapFactoryAddress = "swap-factory-address"
optionNameSwapInitialDeposit = "swap-initial-deposit"
optionNameSwapEnable = "swap-enable"
optionNameChequebookEnable = "chequebook-enable"
optionNameSwapDeploymentGasPrice = "swap-deployment-gas-price"
optionNameFullNode = "full-node"
optionNamePostageContractAddress = "postage-stamp-address"
optionNamePostageContractStartBlock = "postage-stamp-start-block"
optionNamePriceOracleAddress = "price-oracle-address"
optionNameRedistributionAddress = "redistribution-address"
optionNameStakingAddress = "staking-address"
optionNameBlockTime = "block-time"
optionWarmUpTime = "warmup-time"
optionNameMainNet = "mainnet"
optionNameRetrievalCaching = "cache-retrieval"
optionNameDevReserveCapacity = "dev-reserve-capacity"
optionNameResync = "resync"
optionNamePProfBlock = "pprof-profile"
optionNamePProfMutex = "pprof-mutex"
optionNameStaticNodes = "static-nodes"
optionNameAllowPrivateCIDRs = "allow-private-cidrs"
optionNameSleepAfter = "sleep-after"
optionNameRestrictedAPI = "restricted"
optionNameTokenEncryptionKey = "token-encryption-key"
optionNameAdminPasswordHash = "admin-password"
optionNameUsePostageSnapshot = "use-postage-snapshot"
optionNameStorageIncentivesEnable = "storage-incentives-enable"
optionNameStateStoreCacheCapacity = "statestore-cache-capacity"
optionNameTargetNeighborhood = "target-neighborhood"
optionNameNeighborhoodSuggester = "neighborhood-suggester"
optionNameWhitelistedWithdrawalAddress = "withdrawal-addresses-whitelist"
)

// nolint:gochecknoinits
Expand Down Expand Up @@ -302,6 +304,8 @@ func (c *command) setAllFlags(cmd *cobra.Command) {
cmd.Flags().Bool(optionNameStorageIncentivesEnable, true, "enable storage incentives feature")
cmd.Flags().Uint64(optionNameStateStoreCacheCapacity, 100_000, "lru memory caching capacity in number of statestore entries")
cmd.Flags().String(optionNameTargetNeighborhood, "", "neighborhood to target in binary format (ex: 111111001) for mining the initial overlay")
cmd.Flags().String(optionNameNeighborhoodSuggester, "https://api.swarmscan.io/v1/network/neighborhoods/suggestion", "suggester for target neighborhood")
cmd.Flags().StringSlice(optionNameWhitelistedWithdrawalAddress, []string{}, "withdrawal target addresses")
}

func newLogger(cmd *cobra.Command, verbosity string) (log.Logger, error) {
Expand Down
2 changes: 1 addition & 1 deletion cmd/bee/cmd/cmd_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
"os"
"testing"

"github.com/ethersphere/bee/cmd/bee/cmd"
"github.com/ethersphere/bee/v2/cmd/bee/cmd"
)

var homeDir string
Expand Down
10 changes: 5 additions & 5 deletions cmd/bee/cmd/db.go
Original file line number Diff line number Diff line change
Expand Up @@ -18,11 +18,11 @@ import (
"strings"
"time"

"github.com/ethersphere/bee/pkg/node"
"github.com/ethersphere/bee/pkg/postage"
"github.com/ethersphere/bee/pkg/storage"
"github.com/ethersphere/bee/pkg/storer"
"github.com/ethersphere/bee/pkg/swarm"
"github.com/ethersphere/bee/v2/pkg/node"
"github.com/ethersphere/bee/v2/pkg/postage"
"github.com/ethersphere/bee/v2/pkg/storage"
"github.com/ethersphere/bee/v2/pkg/storer"
"github.com/ethersphere/bee/v2/pkg/swarm"
"github.com/spf13/cobra"
)

Expand Down
Loading

0 comments on commit f37a2c2

Please sign in to comment.