Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Test for go-ipfs-cmds#282 and boxo#855 #10738

Closed
wants to merge 1 commit into from
Closed

Test for go-ipfs-cmds#282 and boxo#855 #10738

wants to merge 1 commit into from
+118 −111

Conversation

PsychoPunkSage
Copy link
Contributor

This is a companion PR for ipfs/boxo#855 and ipfs/go-ipfs-cmds#282 that adds an HTTP RPC test for content blocking functionality.

addresses: ipfs/boxo#855 (review)

Description

[WIP] This PR adds a new test case in test/cli/content_blocking_test.go to verify that content blocking works correctly over the HTTP API. The test ensures that:

Blocked content returns the expected HTTP status code (410 Gone)
The response body contains the appropriate error message

[WIP] This demonstrates that the new error handling implemented in the companion PRs (replacing string-based checks with proper error types) maintains compatibility with existing functionality while providing a cleaner approach for future development.
The test directly verifies the HTTP response code, which is critical for ensuring our APIs behave as expected when content is blocked.

@PsychoPunkSage
fix::> Added my repo depn
Loading
Loading status checks…
9dafae3 
Signed-off-by: Abhinav Prakash <[email protected]>
@PsychoPunkSage PsychoPunkSage requested a review from a team as a code owner February 28, 2025 04:27
@PsychoPunkSage
Copy link
Contributor Author

PsychoPunkSage commented Feb 28, 2025 •
edited
Loading

hi @lidel
I made a companion PR as u requested here ipfs/boxo#855 (review)

I noticed the the tests in content_blocking_test.go already fails (w/o introducing the changes I made)

Logs (w/o my changes)

go test content_blocking_test.go -v

=== RUN   TestContentBlocking
2025-02-28T10:05:19.887+0530	PANIC	testharness	harness/node.go:441	node 0 with peer ID 12D3KooWMVXzptNjhw9syDHrYrhz4tQUxDyqY3SmvWHi8pDxDHry failed to come online: 
2025-02-28T10:04:59.932+0530	ERROR	nopfs	[email protected]/blocker.go:29	error opening and processing /tmp/2028115846/.nodes/0/denylists/test.deny: too many open files
2025-02-28T10:04:59.932+0530	ERROR	core	core/builder.go:158	constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".Online.IpnsRepublisher.func6 (github.com/ipfs/kubo/core/node/ipns.go:49): failed to build namesys.NameSystem: could not build arguments for function "github.com/ipfs-shipyard/nopfs/ipfs".WrapNameSystem (github.com/ipfs-shipyard/nopfs/[email protected]/namesys.go:22): failed to build *nopfs.Blocker: received non-nil error from function "github.com/ipfs/kubo/plugin/plugins/nopfs".(*nopfsPlugin).Options.MakeBlocker.func1 (github.com/ipfs/kubo/plugin/plugins/nopfs/nopfs.go:45): too many open files
Error: constructing the node (see log for full detail): too many open files


Initializing daemon...
Kubo version: 0.34.0-dev-3031cf379-dirty
Repo version: 16
System version: amd64/linux
Golang version: go1.23.6
PeerID: 12D3KooWMVXzptNjhw9syDHrYrhz4tQUxDyqY3SmvWHi8pDxDHry


--- FAIL: TestContentBlocking (20.47s)
panic: node 0 with peer ID 12D3KooWMVXzptNjhw9syDHrYrhz4tQUxDyqY3SmvWHi8pDxDHry failed to come online: 
	2025-02-28T10:04:59.932+0530	ERROR	nopfs	[email protected]/blocker.go:29error opening and processing /tmp/2028115846/.nodes/0/denylists/test.deny: too many open files
	2025-02-28T10:04:59.932+0530	ERROR	core	core/builder.go:158	constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".Online.IpnsRepublisher.func6 (github.com/ipfs/kubo/core/node/ipns.go:49): failed to build namesys.NameSystem: could not build arguments for function "github.com/ipfs-shipyard/nopfs/ipfs".WrapNameSystem (github.com/ipfs-shipyard/nopfs/[email protected]/namesys.go:22): failed to build *nopfs.Blocker: received non-nil error from function "github.com/ipfs/kubo/plugin/plugins/nopfs".(*nopfsPlugin).Options.MakeBlocker.func1 (github.com/ipfs/kubo/plugin/plugins/nopfs/nopfs.go:45): too many open files
	Error: constructing the node (see log for full detail): too many open files
	
	
	Initializing daemon...
	Kubo version: 0.34.0-dev-3031cf379-dirty
	Repo version: 16
	System version: amd64/linux
	Golang version: go1.23.6
	PeerID: 12D3KooWMVXzptNjhw9syDHrYrhz4tQUxDyqY3SmvWHi8pDxDHry
	
	 [recovered]
	panic: node 0 with peer ID 12D3KooWMVXzptNjhw9syDHrYrhz4tQUxDyqY3SmvWHi8pDxDHry failed to come online: 
	2025-02-28T10:04:59.932+0530	ERROR	nopfs	[email protected]/blocker.go:29error opening and processing /tmp/2028115846/.nodes/0/denylists/test.deny: too many open files
	2025-02-28T10:04:59.932+0530	ERROR	core	core/builder.go:158	constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".Online.IpnsRepublisher.func6 (github.com/ipfs/kubo/core/node/ipns.go:49): failed to build namesys.NameSystem: could not build arguments for function "github.com/ipfs-shipyard/nopfs/ipfs".WrapNameSystem (github.com/ipfs-shipyard/nopfs/[email protected]/namesys.go:22): failed to build *nopfs.Blocker: received non-nil error from function "github.com/ipfs/kubo/plugin/plugins/nopfs".(*nopfsPlugin).Options.MakeBlocker.func1 (github.com/ipfs/kubo/plugin/plugins/nopfs/nopfs.go:45): too many open files
	Error: constructing the node (see log for full detail): too many open files
	
	
	Initializing daemon...
	Kubo version: 0.34.0-dev-3031cf379-dirty
	Repo version: 16
	System version: amd64/linux
	Golang version: go1.23.6
	PeerID: 12D3KooWMVXzptNjhw9syDHrYrhz4tQUxDyqY3SmvWHi8pDxDHry
	
	

goroutine 19 [running]:
testing.tRunner.func1.2({0x1195c60, 0xc000059d10})
	/usr/local/go/src/testing/testing.go:1632 +0x230
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1635 +0x35e
panic({0x1195c60?, 0xc000059d10?})
	/usr/local/go/src/runtime/panic.go:785 +0x132
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x0?, 0x0?, {0x0?, 0x0?, 0xc000112140?})
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:196 +0x54
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0001301a0, {0x0, 0x0, 0x0})
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:262 +0x24e
go.uber.org/zap.(*SugaredLogger).log(0xc000427f20, 0x4, {0x1434619?, 0x1a?}, {0xc0003dd8c8?, 0x1?, 0x1?}, {0x0, 0x0, 0x0})
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/sugar.go:355 +0xec
go.uber.org/zap.(*SugaredLogger).Panicf(...)
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/sugar.go:229
github.com/ipfs/kubo/test/cli/harness.(*Node).WaitOnAPI(0xc000441180, {0x0, 0x0})
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/harness/node.go:441 +0x2f9
github.com/ipfs/kubo/test/cli/harness.(*Node).StartDaemonWithReq(0xc000441180, {{0x0, 0x0}, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/harness/node.go:243 +0x35b
github.com/ipfs/kubo/test/cli/harness.(*Node).StartDaemon(...)
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/harness/node.go:248
command-line-arguments.TestContentBlocking(0xc000402ea0)
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/content_blocking_test.go:78 +0xa65
testing.tRunner(0xc000402ea0, 0x16aa0a8)
	/usr/local/go/src/testing/testing.go:1690 +0xf4
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1743 +0x390
FAIL	command-line-arguments	20.489s
FAIL
Logs (with my changes)

go test content_blocking_test.go -v

=== RUN   TestContentBlocking
2025-02-28T10:11:57.636+0530	PANIC	testharness	harness/node.go:441	node 0 with peer ID 12D3KooWD29G784dKQr3xy7RZPxWTnX7Gmhh5w6cCKf3SUiLcrsw failed to come online: 
2025-02-28T10:11:37.676+0530	ERROR	nopfs	[email protected]/blocker.go:29	error opening and processing /tmp/3886790312/.nodes/0/denylists/test.deny: too many open files
2025-02-28T10:11:37.676+0530	ERROR	core	core/builder.go:158	constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".Online.IpnsRepublisher.func6 (github.com/ipfs/kubo/core/node/ipns.go:49): failed to build namesys.NameSystem: could not build arguments for function "github.com/ipfs-shipyard/nopfs/ipfs".WrapNameSystem (github.com/ipfs-shipyard/nopfs/[email protected]/namesys.go:22): failed to build *nopfs.Blocker: received non-nil error from function "github.com/ipfs/kubo/plugin/plugins/nopfs".(*nopfsPlugin).Options.MakeBlocker.func1 (github.com/ipfs/kubo/plugin/plugins/nopfs/nopfs.go:45): too many open files
Error: constructing the node (see log for full detail): too many open files


Initializing daemon...
Kubo version: 0.34.0-dev-3031cf379-dirty
Repo version: 16
System version: amd64/linux
Golang version: go1.23.6
PeerID: 12D3KooWD29G784dKQr3xy7RZPxWTnX7Gmhh5w6cCKf3SUiLcrsw


--- FAIL: TestContentBlocking (20.48s)
panic: node 0 with peer ID 12D3KooWD29G784dKQr3xy7RZPxWTnX7Gmhh5w6cCKf3SUiLcrsw failed to come online: 
	2025-02-28T10:11:37.676+0530	ERROR	nopfs	[email protected]/blocker.go:29error opening and processing /tmp/3886790312/.nodes/0/denylists/test.deny: too many open files
	2025-02-28T10:11:37.676+0530	ERROR	core	core/builder.go:158	constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".Online.IpnsRepublisher.func6 (github.com/ipfs/kubo/core/node/ipns.go:49): failed to build namesys.NameSystem: could not build arguments for function "github.com/ipfs-shipyard/nopfs/ipfs".WrapNameSystem (github.com/ipfs-shipyard/nopfs/[email protected]/namesys.go:22): failed to build *nopfs.Blocker: received non-nil error from function "github.com/ipfs/kubo/plugin/plugins/nopfs".(*nopfsPlugin).Options.MakeBlocker.func1 (github.com/ipfs/kubo/plugin/plugins/nopfs/nopfs.go:45): too many open files
	Error: constructing the node (see log for full detail): too many open files
	
	
	Initializing daemon...
	Kubo version: 0.34.0-dev-3031cf379-dirty
	Repo version: 16
	System version: amd64/linux
	Golang version: go1.23.6
	PeerID: 12D3KooWD29G784dKQr3xy7RZPxWTnX7Gmhh5w6cCKf3SUiLcrsw
	
	 [recovered]
	panic: node 0 with peer ID 12D3KooWD29G784dKQr3xy7RZPxWTnX7Gmhh5w6cCKf3SUiLcrsw failed to come online: 
	2025-02-28T10:11:37.676+0530	ERROR	nopfs	[email protected]/blocker.go:29error opening and processing /tmp/3886790312/.nodes/0/denylists/test.deny: too many open files
	2025-02-28T10:11:37.676+0530	ERROR	core	core/builder.go:158	constructing the node: could not build arguments for function "github.com/ipfs/kubo/core/node".Online.IpnsRepublisher.func6 (github.com/ipfs/kubo/core/node/ipns.go:49): failed to build namesys.NameSystem: could not build arguments for function "github.com/ipfs-shipyard/nopfs/ipfs".WrapNameSystem (github.com/ipfs-shipyard/nopfs/[email protected]/namesys.go:22): failed to build *nopfs.Blocker: received non-nil error from function "github.com/ipfs/kubo/plugin/plugins/nopfs".(*nopfsPlugin).Options.MakeBlocker.func1 (github.com/ipfs/kubo/plugin/plugins/nopfs/nopfs.go:45): too many open files
	Error: constructing the node (see log for full detail): too many open files
	
	
	Initializing daemon...
	Kubo version: 0.34.0-dev-3031cf379-dirty
	Repo version: 16
	System version: amd64/linux
	Golang version: go1.23.6
	PeerID: 12D3KooWD29G784dKQr3xy7RZPxWTnX7Gmhh5w6cCKf3SUiLcrsw
	
	

goroutine 22 [running]:
testing.tRunner.func1.2({0x11d77e0, 0xc000508270})
	/usr/local/go/src/testing/testing.go:1632 +0x230
testing.tRunner.func1()
	/usr/local/go/src/testing/testing.go:1635 +0x35e
panic({0x11d77e0?, 0xc000508270?})
	/usr/local/go/src/runtime/panic.go:785 +0x132
go.uber.org/zap/zapcore.CheckWriteAction.OnWrite(0x0?, 0x0?, {0x0?, 0x0?, 0xc000434400?})
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:196 +0x54
go.uber.org/zap/zapcore.(*CheckedEntry).Write(0xc0001ba5b0, {0x0, 0x0, 0x0})
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/zapcore/entry.go:262 +0x24e
go.uber.org/zap.(*SugaredLogger).log(0xc0003b6020, 0x4, {0x14899d6?, 0x1a?}, {0xc00005d8c8?, 0x1?, 0x1?}, {0x0, 0x0, 0x0})
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/sugar.go:355 +0xec
go.uber.org/zap.(*SugaredLogger).Panicf(...)
	/home/psychopunk_sage/go/pkg/mod/go.uber.org/[email protected]/sugar.go:229
github.com/ipfs/kubo/test/cli/harness.(*Node).WaitOnAPI(0xc0003a2770, {0x0, 0x0})
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/harness/node.go:441 +0x2f9
github.com/ipfs/kubo/test/cli/harness.(*Node).StartDaemonWithReq(0xc0003a2770, {{0x0, 0x0}, {0x0, 0x0, 0x0}, {0x0, 0x0, 0x0}, 0x0, ...}, ...)
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/harness/node.go:243 +0x35b
github.com/ipfs/kubo/test/cli/harness.(*Node).StartDaemon(...)
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/harness/node.go:248
command-line-arguments.TestContentBlocking(0xc0003be000)
	/home/psychopunk_sage/dev/OpenSource/kubo/test/cli/content_blocking_test.go:78 +0xa65
testing.tRunner(0xc0003be000, 0x16ffe18)
	/usr/local/go/src/testing/testing.go:1690 +0xf4
created by testing.(*T).Run in goroutine 1
	/usr/local/go/src/testing/testing.go:1743 +0x390
FAIL	command-line-arguments	20.496s
FAIL

Well there is no difference in Errors (if that's what we want to prove)

@PsychoPunkSage
Copy link
Contributor Author

My initial file limit was 1024
so I changed it to 8192 <ulimit -n 8192> still not working.... what should be the limit??

I even cleared all the processes that might be holding open files.

@guillaumemichel guillaumemichel marked this pull request as draft March 4, 2025 10:07
@lidel lidel mentioned this pull request Mar 11, 2025
Closed
@lidel lidel changed the title Test for Content Blocking Test for go-ipfs-cmds#282 and boxo#855 Mar 11, 2025
@lidel
Copy link
Member

lidel commented Mar 11, 2025

@lidel
Copy link
Member

lidel commented Mar 28, 2025

ipfs/boxo#855 (comment)

@lidel lidel closed this Mar 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
2 participants
@PsychoPunkSage @lidel