Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"ninja: build stopped: subcommand failed." - which subcommand ? No clue in log. #2378

Open
JVD66 opened this issue Jan 20, 2024 · 7 comments

Comments

@JVD66
Copy link

JVD66 commented Jan 20, 2024

Please, improve ninja to say WHAT subcommand failed ; simply exiting with
"ninja: build stopped: subcommand failed."
is useless - it doesn't tell people WHAT failed.

My log file, for a GCC build, shows NO error messages from GCC, so I'm stuck.

I have to run the whole build with 'strace -s8192 -f -e trace=execve' and tediously
analyse this log to find which sub-processes / commands have failed.

This is unacceptable - I'd recommend converting projects from ninja use to plain GNU Make
use or CMake use because of this issue (and many others).

@digit-google
Copy link
Contributor

You may be misunderstood. Ninja already prints the subcommand which failed before this message (as well as the combined stdout + stderr of that command). You can verify that by writing the following to a build.ninja file the invoking Ninja in the same directory:

rule foo
  command = echo "to_stdout"; echo >&2 "to_stderr"; exit 1

build out: foo

You will get:

[1/1] echo "to_stdout"; echo >&2 "to_stderr"; exit 1
FAILED: out 
echo "to_stdout"; echo >&2 "to_stderr"; exit 1
to_stdout
to_stderr
ninja: build stopped: subcommand failed.

However, the failed command + its outputs are sent to stderr, while the "ninja: build stopped: subcommand failed" message is sent to stdout.

Did you redirect stderr to a different file before invoking Ninja? That would explain what you're are seeing.

@johnkrah
Copy link

@digit-google the example you've given is clearly true but I think it is too simple to attempt to simulate the user feedback. Problem that @JVD66 is describing and I can reproduce is something like the following:

  1. have a build with many targets and use a high number of concurrent tasks, e.g. -j 300
  2. one of those targets fails (naturally or synthetically)
  3. ninja pipes stdout and stderr to console/log for any and all tasks that have started
    3.1. and consider the case where there is non-trivial amount of info printed for many targets
  4. the failing target is usually not the last target to complete
    4.1. almost perforce since the failing target will signal to ninja to stop and other subcommands will either stop after signaled or stop when naturally complete
  5. this results in the information necessary to debug the problem is far away from the end of console/log

it's also true that with knowledge that ninja will output the string FAILED: out one can search regular expression ^FAILED: (?<target>.+)$ or whatever other kind of string search to find that line and the subommand's actual output should be near it as well.

requested feature improvement is for ninja to output a summary statement identifying which subcommand failed. using the same example you supplied, I'd add that the desired output is:

ninja: build stopped: subcommand failed: out.

or simpler alternative that is a constant log message and leaves searching to the user but gives a gentle nudge in the right direction:

ninja: build stopped: subcommand failed. Look for "FAILED: <build target name>" above.

@zougloub
Copy link

If not re-printing failing commands, please remind us to look for FAILED when a failure occurs. This is really a PITA when compiling with a large number of jobs.

@summerfind
Copy link

Hi,
I just build hello_world and got error: FAILED: hello_world.elf ---- collect2.exe: error: ld returned 1 exit status --- ninja: build stopped: subcommand failed.
how to fix?
Thanks

@rustyhowell-c4
Copy link

Because ninja output can be very large, it is difficult to find the error message in an active terminal window. It would be great if ninja would write the entire log to a log file all the time. Yes I know stdout and stderr can be redirected, but so often it's a pain to rerun ninja just to capture the error.

Another option would be to write just the stdout of the failing thread to a log file. That would be helpful as well.

@flexibeast
Copy link

requested feature improvement is for ninja to output a summary statement identifying which subcommand failed

To which i'd add: "together with the error message and/or code associated with the failure." It seems make-work to me to force people to speculate as to what the reason(s) for the failure might be if that information has been provided by the subcommand, and makes it more difficult to assist people who are experiencing compilation failures.

@vvuk
Copy link

vvuk commented Sep 7, 2024

Was surprised by this behaviour and came across this. ninja already buffers command output, and it knows what command(s) failed the build. Seems like it should be straightforward to just print the failing command(s) + their output again just before exit, as if they were the ones that happened to execute last?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants