Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Spurious cctrace errors with long command lines #9

Open
phsmenon opened this issue Dec 14, 2018 · 3 comments
Open

Spurious cctrace errors with long command lines #9

phsmenon opened this issue Dec 14, 2018 · 3 comments
Labels
bug Something isn't working

Comments

@phsmenon
Copy link
Contributor

For long command lines cctrace appears to see/check only the first 4095 characters or so. This seems to be causing bogus "missing argument" errors. For example, see the error on line 978 in the attached log file; the command line appears to be cut-off towards the end.

When I build with fewer flags (which causes the command line to shrink somewhat), the error seems to go away.

I built the variant set based on the latest VGT and the postgres ATD included with it. For reference, I have attached the cctrace policy file I used as well as the build and command logs.

cctrace-postgres-with-long-line-error.log

variant-21-logs-and-policy.zip

@thedataking thedataking added the bug Something isn't working label Dec 15, 2018
@thedataking
Copy link
Collaborator

thedataking commented Dec 15, 2018

Thanks for the detailed bug report @phsmenon! It seems like you are building with -j32 so it is possible that the sysdig kernel ring buffer can't keep up or maybe we're not processing them fast enough in userland. As far as I understand, sysdig is geared toward production use meaning that they will drop events rather than slowing down the system being monitored.

Would it be easy for you to try a build with the full set of flags but with less parallelism?

@phsmenon
Copy link
Contributor Author

I tried the build with no parallelism. But it seems to exhibit the same behavior. I have attached the cctrace logs as well as the build logs.

Let me know if I you want me to try anything else.

cctrace-postgres-with-long-line-error-2.log

variant-21-logs-and-policy.zip

@thedataking
Copy link
Collaborator

I can reproduce the issue using a small, single-threaded test script. It looks like we're hitting a sysdig limitation. According to this issue, sysdig does not handle arguments longer than one page size.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants