Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BufferFull error on specific store path #19

Open
mikepurvis opened this issue Nov 29, 2022 · 8 comments · May be fixed by #20
Open

BufferFull error on specific store path #19

mikepurvis opened this issue Nov 29, 2022 · 8 comments · May be fixed by #20

Comments

@mikepurvis
Copy link

mikepurvis commented Nov 29, 2022 •
edited
Loading

An empty response (no headers, nothing) comes back, and the log shows:

10.26.14.222 - - [29/Nov/2022:17:34:55 +0000] "GET /y94ivgf0ygn86fwsqqhpb83lk7bhprlb.narinfo HTTP/1.1" 200 22967 "" "curl/7.81.0"
toBufIOWith: BufferFull: minSize
CallStack (from HasCallStack):
  error, called at ./Network/Wai/Handler/Warp/IO.hs:23:36 in warp-3.3.21-Fi06NMufirnA6UXRcnfV9n:Network.Wai.Handler.Warp.IO

Regular nix-serve and harmonia have no problem serving this narinfo, though the References list is quite extensive— the overall size of the narinfo response is almost 23k bytes.
@Gabriella439
Copy link
Collaborator

If it's not sensitive, could you upload a NAR for the path in question by running:

$ nix-store --dump /nix/store/y94ivgf0ygn86fwsqqhpb83lk7bhprlb* > example.nar

That would help us reproduce the issue locally

@mikepurvis
Copy link
Author

Unfortunately I'm not able to upload it, but the length of the References field was definitely the most significant and interesting aspect:

$ curl http://hydra.clearpath.ai/cache/y94ivgf0ygn86fwsqqhpb83lk7bhprlb.narinfo | grep References | wc
      1     455   22507

Maybe a trivial derivation that references a few hundred things in nixpkgs would repro the issue?

@angerman angerman mentioned this issue Dec 6, 2022
Closed
@angerman
Copy link

angerman commented Dec 7, 2022

Maybe it's related to this: yesodweb/wai#894, and a simple byte string update fixes it. This to me happens with a [("Content-Type","text/x-nix-narinfo"),("Content-Length","27466")]. The references line alone is ~17kb.

@angerman
Copy link

angerman commented Dec 7, 2022

And this is the relevant haskell/bytestring#538 issue. Until byte string 0.11.4 is released, I guess the best is to either s-r-p a byte string, or downgrade to say ghc 8.10.7.

angerman added a commit to angerman/nix-serve-ng that referenced this issue Dec 7, 2022
@angerman
Fix aristanetworks#19
10bae75 
This fixes aristanetworks#19 by downgrading the compiler to 8.10 (I still don't think 9.0 should be used in production).
bytestring < 0.11.4 (which as of today has not been released to https://hackage.haskell.org/package/bytestring),
is broken, as per haskell/bytestring#538. This in turn shows up in yesodweb/wai#894, and ultimately causes aristanetworks#19.
@angerman angerman linked a pull request Dec 7, 2022 that will close this issue
Open
@mikatammi
Copy link

I'm also seeing this:

warning: error: unable to download 'http://172.18.8.142:5000/bqnisnimwjiaki9376232b1jlydz3qn2.narinfo': Server returned nothing (no headers, no data) (52); retrying in 334 ms
warning: error: unable to download 'http://172.18.8.142:5000/bqnisnimwjiaki9376232b1jlydz3qn2.narinfo': Server returned nothing (no headers, no data) (52); retrying in 543 ms
warning: error: unable to download 'http://172.18.8.142:5000/bqnisnimwjiaki9376232b1jlydz3qn2.narinfo': Server returned nothing (no headers, no data) (52); retrying in 1024 ms
warning: error: unable to download 'http://172.18.8.142:5000/bqnisnimwjiaki9376232b1jlydz3qn2.narinfo': Server returned nothing (no headers, no data) (52); retrying in 2440 ms
error: unable to download 'http://172.18.8.142:5000/bqnisnimwjiaki9376232b1jlydz3qn2.narinfo': Server returned nothing (no headers, no data) (52)

The package in question is "runtime-deps", I get this while running nixos-rebuild

In the server's logs:

May 30 15:05:10 nixos-ryzen nix-daemon[2231718]: accepted connection from pid 63210, user nix-serve
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]: 172.18.8.152 - - [30/May/2023:15:05:10 +0300] "GET /bqnisnimwjiaki9376232b1jlydz3qn2.narinfo HTTP/1.1" 200 28219 "" "curl/7.86.0 Nix/2.11.1"
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]: toBufIOWith: BufferFull: minSize
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]: CallStack (from HasCallStack):
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]:   error, called at ./Network/Wai/Handler/Warp/IO.hs:23:36 in warp-3.3.21-Fi06NMufirnA6UXRcnfV9n:Network.Wai.Handler.Warp.IO
May 30 15:05:10 nixos-ryzen hydra-queue-runner[2231957]: checking the queue for builds > 822...
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]: 172.18.8.152 - - [30/May/2023:15:05:10 +0300] "GET /bqnisnimwjiaki9376232b1jlydz3qn2.narinfo HTTP/1.1" 200 28219 "" "curl/7.86.0 Nix/2.11.1"
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]: toBufIOWith: BufferFull: minSize
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]: CallStack (from HasCallStack):
May 30 15:05:10 nixos-ryzen nix-serve-start[63210]:   error, called at ./Network/Wai/Handler/Warp/IO.hs:23:36 in warp-3.3.21-Fi06NMufirnA6UXRcnfV9n:Network.Wai.Handler.Warp.IO
May 30 15:05:11 nixos-ryzen nix-serve-start[63210]: 172.18.8.152 - - [30/May/2023:15:05:10 +0300] "GET /bqnisnimwjiaki9376232b1jlydz3qn2.narinfo HTTP/1.1" 200 28219 "" "curl/7.86.0 Nix/2.11.1"
May 30 15:05:11 nixos-ryzen nix-serve-start[63210]: toBufIOWith: BufferFull: minSize
May 30 15:05:11 nixos-ryzen nix-serve-start[63210]: CallStack (from HasCallStack):
May 30 15:05:11 nixos-ryzen nix-serve-start[63210]:   error, called at ./Network/Wai/Handler/Warp/IO.hs:23:36 in warp-3.3.21-Fi06NMufirnA6UXRcnfV9n:Network.Wai.Handler.Warp.IO
May 30 15:05:12 nixos-ryzen nix-serve-start[63210]: 172.18.8.152 - - [30/May/2023:15:05:12 +0300] "GET /bqnisnimwjiaki9376232b1jlydz3qn2.narinfo HTTP/1.1" 200 28219 "" "curl/7.86.0 Nix/2.11.1"
May 30 15:05:12 nixos-ryzen nix-serve-start[63210]: toBufIOWith: BufferFull: minSize
May 30 15:05:12 nixos-ryzen nix-serve-start[63210]: CallStack (from HasCallStack):
May 30 15:05:12 nixos-ryzen nix-serve-start[63210]:   error, called at ./Network/Wai/Handler/Warp/IO.hs:23:36 in warp-3.3.21-Fi06NMufirnA6UXRcnfV9n:Network.Wai.Handler.Warp.IO
May 30 15:05:15 nixos-ryzen nix-serve-start[63210]: 172.18.8.152 - - [30/May/2023:15:05:15 +0300] "GET /bqnisnimwjiaki9376232b1jlydz3qn2.narinfo HTTP/1.1" 200 28219 "" "curl/7.86.0 Nix/2.11.1"
May 30 15:05:15 nixos-ryzen nix-serve-start[63210]: toBufIOWith: BufferFull: minSize
May 30 15:05:15 nixos-ryzen nix-serve-start[63210]: CallStack (from HasCallStack):
May 30 15:05:15 nixos-ryzen nix-serve-start[63210]:   error, called at ./Network/Wai/Handler/Warp/IO.hs:23:36 in warp-3.3.21-Fi06NMufirnA6UXRcnfV9n:Network.Wai.Handler.Warp.IO

@mikatammi
Copy link

Moved to @Mic92 's fork for now https://github.com/Mic92/nix-serve-ng/commits/fix-build
Until I change this system to a flake-based configuration where I can override nixpkgs in the inputs

@Gabriella439
Copy link
Collaborator

Unfortunately I no longer have the commit bit on this repository so I can't merge any fixes at the moment (cc: @jsoo1)

@jsoo1
Copy link
Collaborator

jsoo1 commented Jun 12, 2023

I am so sorry! This has totally slipped my mind. I can prioritize review now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fix #19 angerman/nix-serve-ng
5 participants
@angerman @mikatammi @mikepurvis @Gabriella439 @jsoo1