Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bug: Blight journal is empty #63

Open
F1o0T opened this issue Jun 22, 2023 · 34 comments
Open

bug: Blight journal is empty #63

F1o0T opened this issue Jun 22, 2023 · 34 comments
Labels
bug Something isn't working

Comments

@F1o0T
Copy link

F1o0T commented Jun 22, 2023

Hello,

Today, I have started to use MATE, I was able to compile and go through the notes.c example. However, I wasn't able to compile my example project that is compiled using 'make' after the Makefile generation through cmake. The Makefile is created under the build dir in the root src dir.

 mate-cli oneshot exampleProject/                                                          
✖ bb83b24f0e4242648ab3f9592ea1e0f7: failed

=======================
EXITED WITH: 2
=======================

=======================
STDOUT:

=======================

=======================
STDERR:
make: /usr/bin/cmake: Command not found
make: *** [Makefile:4932: cmake_check_build_system] Error 127

=======================
Although, I already have cmake at /usr/bin/cmake

whereis cmake                                                                     
cmake: **/usr/bin/cmake** /usr/lib/x86_64-linux-gnu/cmake /usr/lib/cmake /usr/share/cmake /usr/share/man/man1/cmake.1.gz /usr/src/googletest/googletest/cmake

I have also tried to creat a tar out of the exampleProject/ dir
tar czf exampleProject.tar.gz exampleProject/
mate-cli artifact create compile-target:tarball exampleProject.tar.gz
mate-cli compile create --wait --make-targets exampleProject.elf --artifact-id 6226e34a987942589c5c9cadb089c9ea

Still, it didn't work!

Any idea what I could did wrong, thank you!

@thinkmoore
Copy link
Contributor

By default, compilation takes place in a docker image that does not have access to your local environment and cmake is not present. Two workarounds are:

  1. Edit the makefile to install the necessary dependencies before the step that breaks
  2. Use the more detailed mate-cli workflow to specify a container to use for the build (https://galoisinc.github.io/MATE/cli-overview.html#a-basic-workflow-with-mate-cli, https://galoisinc.github.io/MATE/cli.html#mate-cli-compile-create)

Unfortunately, the documentation for the latter is currently lacking...

@F1o0T
Copy link
Author

F1o0T commented Jun 23, 2023

Hi @thinkmoore,

Thanks for your reply, I have modified the Makefile to install cmake, and now it is able to see cmake, but I think command mate-cli is also still not mounting the sourcr directory into the container.

Now I get the error:

STDERR:
CMake Error: The source directory "/home/user/projects/testMate/exampleProject" does not exist.
Specify --help for usage, or press the help button on the CMake GUI.
make: *** [Makefile:4933: cmake_check_build_system] Error 1

May I ask if you can give some hints about how to use the more detailed mate-cli workflow to specify a container to use for the build, since as you said, the link you shared doesn't have enough information about that.

@F1o0T
Copy link
Author

F1o0T commented Jun 23, 2023

I have fixed the error of not being able to find the source directoy by mouning that directory into the container using the docker-compose.yml file by adding an entry to the executer under the volume section.

Now, I am facing another issue, my project is compiling successfully, but the build command is not showing any output. I understand that the build command is building the CPG graph. However, it is not showing anything.

image

Is it maybe because I am compiling for riscv not x86?

Thanks in advance for your support.

@thinkmoore
Copy link
Contributor

0 builds completed successfully makes me wonder if something went wrong with the build interposition and we're not seeing any bitcode files. Can you run the first command here https://galoisinc.github.io/MATE/debugging-builds.html#debugging-a-build and see if there are any bitcode_artifact entries?

You might also check docker service logs mate_executor_1 for errors.

@F1o0T
Copy link
Author

F1o0T commented Jul 2, 2023

Here is the output of docker container logs mate-executer-one

 [2023-07-02 11:12:37,969: INFO/MainProcess] Task mate.tasks.compile.compile_artifact[8449e833-5b2c-4549-942b-54dab6120080] received
 [2023-07-02 11:12:37,970: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7fb71ade7700> (args:('mate.tasks.compile.compile_artifact', '8449e833-5b2c-4549-942b-54dab6120080', {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '8449e833-5b2c-4549-942b-54dab6120080', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '8449e833-5b2c-4549-942b-54dab6120080', 'parent_id': None, 'argsrepr': "('d0a1495933d7400a88cab6310bec5c96', CompileOptions(testbed=None, containerized=False, experimental_embed_bitcode=False, docker_image=None, containerized_infer_build=True, make_targets=None, extra_compiler_flags=[]))", 'kwargsrepr': "{'compilation_id': 'fa0403f986a74d4984d9949918e245b9'}", 'origin': 'gen29@6fb04f90469d', 'ignore_result': False, 'properties': {'content_type': 'application/x-python-serialize', 'content_encoding': 'binary', 'application_headers': {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '8449e833-5b2c-4549-942b-54dab6120080', 'shadow': None, 'eta': None, 'expires': None, 'group':... kwargs:{})
 [2023-07-02 11:12:37,994: DEBUG/ForkPoolWorker-8] http://storage:9000 "GET /artifacts/d0a1495933d7400a88cab6310bec5c96 HTTP/1.1" 200 272442946
 [2023-07-02 11:12:42,450: DEBUG/ForkPoolWorker-8] starting at /tmp/tmpobg741j0/s400; searching ["I have intentially deleted file paths here since I am using it on proprietary codebase"]
 [2023-07-02 11:12:42,450: DEBUG/ForkPoolWorker-8] /tmp/tmpobg741j0/exampleProject does not contain a Makefile; checking the next layer
 [2023-07-02 11:12:42,451: DEBUG/ForkPoolWorker-8] found a makefile in /tmp/tmpobg741j0/exampleProject/build
 [2023-07-02 11:12:42,451: DEBUG/ForkPoolWorker-8] running args=['blight-exec', '--swizzle-path', '--guess-wrapped', '--', 'make']
 [2023-07-02 11:12:42,616: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/084ed03d58a5421199e5737ef10402a6 HTTP/1.1" 200 0
 [2023-07-02 11:12:42,617: ERROR/ForkPoolWorker-8]
 You have encountered a bug in MATE's implementation. Please report it.

 Additional information:
 artifact: given an empty fileobj to upload self.uuid='e601f9e652194a53872c44820731f976' self.kind=<ArtifactKind.BlightJournal: 'blight:journal'>
 [2023-07-02 11:12:42,621: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/e601f9e652194a53872c44820731f976 HTTP/1.1" 200 0
 [2023-07-02 11:12:42,892: INFO/ForkPoolWorker-8] Task mate.tasks.compile.compile_artifact[8449e833-5b2c-4549-942b-54dab6120080] succeeded in 4.921525783999641s: None

When I compare that log output with the log output of the notes.c provided example, it seems that the get-bc command is not called since an error occured while executing blight-exec.

@langston-barrett langston-barrett added the bug Something isn't working label Jul 3, 2023
@langston-barrett
Copy link
Collaborator

Here is that assertion.

@langston-barrett langston-barrett changed the title Unable to comile a Make based project bug: Blight journal is empty Jul 3, 2023
@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

Hi @langston-barrett,

I am still not sure how to work-around that issue, I have prepared a very simple cmake/make based project with the notes.c source file only. It seems that Mate/blight is not able to intercept the build commands so that it creates bitcode.

image

@woodruffw
Copy link
Collaborator

@xreinheitx could you check which version of blight you're running with? It's unlikely, but this may have been a regression on my part (blight is maintained externally, and has undergone changes since MATE was last released).

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

@woodruffw , I think Mate is using blight which is shipped with the container ghcr.io/galoisinc/mate-dist:main, I am not sure how can I get the version of that one.

@thinkmoore
Copy link
Contributor

I wonder if the problem is that CMAKE is hardcoding a path for cc, etc. and so blight's PATH-based interposition isn't working...

@matteasu
Copy link

@xreinheitx you can run a shell on that specific container and then check the version using pip list
https://kodekloud.com/blog/docker-exec/

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

Hi @matteasu,

Thanks for the hint, the version is 0.0.41

root@7f4a8e195da9:~# pip list | grep blight
blight                 0.0.41

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

@thinkmoore, FYI, I grepped grep -r '$(CC)' . the build dir that CMake used to store the Makefiles, and I couldn't see that the CC is used!

@woodruffw
Copy link
Collaborator

Thanks for the hint, the version is 0.0.41

Thanks for checking -- that version is pretty old, which is good in this case: it's one that was known to work with MATE 🙂

Per what @thinkmoore said: this suggests something weird about the build itself, since CMake should respect CC, CXX, etc. in the blight-instrumented environment. I don't remember the invocation off the top of my head, but I seem to recall there being a mate-cli invocation that'll dumb the logs for each compilation attempt, including any environment variables. That might be a good place to check next.

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

@woodruffw

Do you mean a mate-cli compile option?

mate-cli compile --help
usage: mate-cli compile [-h] {create,get} ...

positional arguments:
  {create,get}
    create      create a compilation
    get         list compilations

options:
  -h, --help    show this help message and exit

And again here is the log that I got using docker container logs mate-executer-1

 [2023-07-10 16:07:28,683: INFO/MainProcess] Task mate.tasks.compile.compile_artifact[01499b65-5038-4c56-abf6-ec3c1fc492dc] received
 [2023-07-10 16:07:28,684: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7fcb0bad0700> (args:('mate.tasks.compile.compile_artifact', '01499b65-5038-4c56-abf6-ec3c1fc492dc', {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '01499b65-5038-4c56-abf6-ec3c1fc492dc', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': '01499b65-5038-4c56-abf6-ec3c1fc492dc', 'parent_id': None, 'argsrepr': "('aaf0467e840e47af9b3e591f5dfe268b', CompileOptions(testbed=None, containerized=False, experimental_embed_bitcode=False, docker_image=None, containerized_infer_build=True, make_targets=None, extra_compiler_flags=[]))", 'kwargsrepr': "{'compilation_id': '270bcb6c99ac41f78133945ee879050e'}", 'origin': 'gen28@c312ff637c87', 'ignore_result': False, 'properties': {'content_type': 'application/x-python-serialize', 'content_encoding': 'binary', 'application_headers': {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': '01499b65-5038-4c56-abf6-ec3c1fc492dc', 'shadow': None, 'eta': None, 'expires': None, 'group':... kwargs:{})
 [2023-07-10 16:07:28,748: DEBUG/ForkPoolWorker-8] http://storage:9000 "GET /artifacts/aaf0467e840e47af9b3e591f5dfe268b HTTP/1.1" 200 33992
 [2023-07-10 16:07:28,786: DEBUG/ForkPoolWorker-8] starting at /tmp/tmpdcsmn_kn/mateExampleCode; searching [PosixPath('/tmp/tmpdcsmn_kn/mateExampleCode/CMakeLists.txt'), PosixPath('/tmp/tmpdcsmn_kn/mateExampleCode/build'), PosixPath('/tmp/tmpdcsmn_kn/mateExampleCode/notes.c')]
 [2023-07-10 16:07:28,786: DEBUG/ForkPoolWorker-8] /tmp/tmpdcsmn_kn/mateExampleCode does not contain a Makefile; checking the next layer
 [2023-07-10 16:07:28,787: DEBUG/ForkPoolWorker-8] found a makefile in /tmp/tmpdcsmn_kn/mateExampleCode/build
 [2023-07-10 16:07:28,788: DEBUG/ForkPoolWorker-8] running args=['blight-exec', '--swizzle-path', '--guess-wrapped', '--', 'make']
 [2023-07-10 16:07:31,678: DEBUG/MainProcess] heartbeat_tick : for connection 7f8209c98a5a4af9983be79384c1cd0c
 [2023-07-10 16:07:31,678: DEBUG/MainProcess] heartbeat_tick : Prev sent/recv: 313/33461, now - 314/33494, monotonic - 80052.570986458, last_heartbeat_sent - 80052.570985486, heartbeat int. - 60 for connection 7f8209c98a5a4af9983be79384c1cd0c
 [2023-07-10 16:07:31,800: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/d3c6ed0bc0f341edbab6c0e777628cca HTTP/1.1" 200 0
 [2023-07-10 16:07:31,801: ERROR/ForkPoolWorker-8]
 You have encountered a bug in MATE's implementation. Please report it.

 Additional information:
 artifact: given an empty fileobj to upload self.uuid='5c221f594d554389bc9aa5b843249db1' self.kind=<ArtifactKind.BlightJournal: 'blight:journal'>

@woodruffw
Copy link
Collaborator

Do you mean a mate-cli compile option?

I believe there's a mate-cli artifact subcommand that you can use to interact with different outputs produced by the compilation, including the dedicated blight logs.

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

@woodruffw I will search for that option.

@thinkmoore, I have tested Mate against an only make simple project (not a cmake-based) with one source file and a Makefile, the Makefile in this case is using $(CC) explicitly. I still see the same issue.

image

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

@woodruffw, I am trying to figure out which option could help me to get more insights with mate-cli artifact, it is not clear. I have usedmate-cli artifact get <ID>but then I got no useful information about what went wrong.

mate-cli artifact --help
usage: mate-cli artifact [-h] {create,dump,get} ...

positional arguments:
  {create,dump,get}
    create           create an artifact
    dump             dump the contents of an artifact
    get              list artifacts

options:
  -h, --help         show this help message and exit
mate-cli artifact get 779dc3b34343471fbfb7a932f12e1b2f
[
    {
        "artifact_id": "779dc3b34343471fbfb7a932f12e1b2f",
        "attributes": {
            "filename": "tmprot_7u_x.tar.gz"
        },
        "build_ids": [],
        "compilation_ids": [
            "d116ac61577d44cf801ee2bbfa77af3b"
        ],
        "has_object": true,
        "kind": "compile-target:tarball"
    }
]

@woodruffw
Copy link
Collaborator

@xreinheitx try this:

mate-cli artifact get --kind compile-output:compile-log

that should dump one or more artifact IDs, which you can then do this work:

mate-cli artifact dump SOME-ID-HERE

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

Hi @woodruffw, thanks a lot for the hint.

Here is the ouput of the dump command for the artifact, I still don't see where is the problem!

mate-cli artifact dump 6e8849874e994aba8779262817201195

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
blight-cc test1.c -o test1.c.o

=======================

=======================
STDERR:
INFO:Entering CC [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
DEBUG:Compile using parsed arguments:
InputList:         [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
InputFiles:        [test1.c]
ObjectFiles:       []
OutputFilename:    test1.c.o
CompileArgs:       [-g3 -grecord-gcc-switches]
LinkArgs:          []
ForbiddenFlags:    []
IsVerbose:         false
IsDependencyOnly:  false
IsPreprocessOnly:  false
IsAssembleOnly:    false
IsAssembly:        false
IsCompileOnly:     false
IsEmitLLVM:        false
IsLTO:             false
IsPrintOnly:       false

DEBUG:buildObjectFile: [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o]
DEBUG:Calling execCmd(clang, [test1.c -o test1.c.o -g3 -grecord-gcc-switches])
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches -emit-llvm -c test1.c -o .test1.c.o.bc] had exitCode 0
DEBUG:execCmd: clang [test1.c -o test1.c.o -g3 -grecord-gcc-switches] had exitCode 0
DEBUG:attachBitcodePathToObject recognized .o as something it can inject into.
DEBUG:execCmd: objcopy [--add-section .llvm_bc=/tmp/gllvm232942747 .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [.test1.c.o -o test1.c.o] had exitCode 0
INFO:LINKING: clang [.test1.c.o -o test1.c.o]
DEBUG:Calling [/opt/mate/libexec/gclang test1.c -o test1.c.o -g3 -grecord-gcc-switches] returned 0

=======================

@woodruffw
Copy link
Collaborator

Hmm -- that looks pretty reasonable, although I don't actually see where the final object file link step occurs: it produces test1.c.o, but doesn't appear to detect test1? Maybe the linker is being invoked manually (rather than through the C compiler), and that's confusing the instrumentation?

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

but doesn't appear to detect test1

I think I confused you because during this troubleshooting, I modified the Makefile to be very simple and to have an explicit CC. Let me list here what I had for the last output.

Here is the source file test1.c
int main() { return 0; }

Here is the current Makefile

all:
	$(CC) test1.c -o test1.c.o

Here is the directory content

>_  ls simpleMakeExample/
Makefile  test1.c

Here is mate-cli executed command
mate-cli oneshot simpleMakeExample

And I got compile-log using your provided instructions above.

mate-cli artifact dump 6e8849874e994aba8779262817201195

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
blight-cc test1.c -o test1.c.o

=======================

=======================
STDERR:
INFO:Entering CC [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
DEBUG:Compile using parsed arguments:
InputList:         [test1.c -o test1.c.o -g3 -grecord-gcc-switches]
InputFiles:        [test1.c]
ObjectFiles:       []
OutputFilename:    test1.c.o
CompileArgs:       [-g3 -grecord-gcc-switches]
LinkArgs:          []
ForbiddenFlags:    []
IsVerbose:         false
IsDependencyOnly:  false
IsPreprocessOnly:  false
IsAssembleOnly:    false
IsAssembly:        false
IsCompileOnly:     false
IsEmitLLVM:        false
IsLTO:             false
IsPrintOnly:       false

DEBUG:buildObjectFile: [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o]
DEBUG:Calling execCmd(clang, [test1.c -o test1.c.o -g3 -grecord-gcc-switches])
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches test1.c -c -o .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [-g3 -grecord-gcc-switches -emit-llvm -c test1.c -o .test1.c.o.bc] had exitCode 0
DEBUG:execCmd: clang [test1.c -o test1.c.o -g3 -grecord-gcc-switches] had exitCode 0
DEBUG:attachBitcodePathToObject recognized .o as something it can inject into.
DEBUG:execCmd: objcopy [--add-section .llvm_bc=/tmp/gllvm232942747 .test1.c.o] had exitCode 0
DEBUG:execCmd: clang [.test1.c.o -o test1.c.o] had exitCode 0
INFO:LINKING: clang [.test1.c.o -o test1.c.o]
DEBUG:Calling [/opt/mate/libexec/gclang test1.c -o test1.c.o -g3 -grecord-gcc-switches] returned 0

=======================

@F1o0T
Copy link
Author

F1o0T commented Jul 10, 2023

@woodruffw With that, I still get 0 build completed successfully.

 mate-cli oneshot simpleMakeExample                                                                                                                                                    
✔ 6825f6b64c5a40d8b7fbab9f78a99c75: compiled
✔ 0 builds completed successfully

@woodruffw
Copy link
Collaborator

Okay, so you're intentionally not producing an executable, right? Just an object file?

In that case, this doesn't work by design: MATE expects a compilation step to produce one or more executables, since the CPG build process fundamentally requires a main or similar entrypoint.

That being said, it's strange that it fails when you do produce a full executable. My only guess is that the linker is somehow being invoked separately, in a way that blight/GLLVM can't detect.

@F1o0T
Copy link
Author

F1o0T commented Jul 11, 2023

Hi @woodruffw,

In the last example in my last message the make file I used is actually generating an executable with an entry point, it is just the name is test1.c.o, but it is an executable which has an entry point main. And still Mate is unable to build the CPG.

all:
	$(CC) test1.c -o test1.c.o

@woodruffw
Copy link
Collaborator

Ah, got it. Just confirming: you see the same behavior when you generate the output file as test1 without the .o suffix, right?

@F1o0T
Copy link
Author

F1o0T commented Jul 11, 2023

Interesting enought to say that when I removed .o from the executable name, it worked!

all:
	$(CC) test1.c -o test1

@F1o0T
Copy link
Author

F1o0T commented Jul 11, 2023

Thenaks @woodruffw for your time! Now we know it worked with that example make project when I removed the .o suffix.

Now, when I get back to the simple Cmake-based project, as you know cmake's job is to automatically generate Makefiles that can build the targets mentioned in the CMakeLists.txt file.

Now the problem is that when I checked the cmake generated Makefiles, I found that cmake is always adding .o at the end of the generated object file before finally doing the linking.

/usr/bin/cc $(C_DEFINES) $(C_INCLUDES) $(C_FLAGS) -MD -MT CMakeFiles/mateExampleCode.elf.dir/notes.c.o -MF CMakeFiles/mateExampleCode.elf.dir/notes.c.o.d -o CMakeFiles/mateExampleCode.elf.dir/notes.c.o -c /home/abdelaziz/projects/testMate/mateExampleCode/notes.c

Here is how the linking is done.

/usr/bin/cc CMakeFiles/mateExampleCode.elf.dir/notes.c.o -o mateExampleCode.elf
With that I still have the 0 build with the simple CMake project.

mate-cli oneshot mateCMakeExampleCode
✔ 8315dbfc5c054b43993f85079605e6f2: compiled
✔ 0 builds completed successfully
[]

I have modified the Makefiles so that it doesn't have .o suffix, and I run mate-cli again, still I got 0 builds

 mate-cli oneshot mateCMakeExampleCode
✔ 8784bedd6c334623811baef6ebfdebc5: compiled
✔ 0 builds completed successfully
[]

And here is the compile-log output.

mate-cli artifact dump efc16c943e70420e96d57fde3c0d854f                            

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
[ 50%] Building C object CMakeFiles/mateExampleCode.elf.dir/notes
[100%] Linking C executable mateExampleCode.elf
[100%] Built target mateExampleCode.elf

=======================

=======================
STDERR:

=======================

@woodruffw
Copy link
Collaborator

No problem! Glad to see we're making progress 🙂

The .elf suffix might also be confusing the instrumentation -- are you able to tweak the build to remove that as well?

@F1o0T
Copy link
Author

F1o0T commented Jul 11, 2023

Unfortuantly, this didn't help.

Dropped .elf suffix from the final executable as followes:

/usr/bin/cc -rdynamic CMakeFiles/mateExampleCode.elf.dir/notes -o mateExampleCode 

And still it didn't work, here is the compile-log.

mate-cli artifact dump feaace818597482cb159a081af265101

=======================
EXITED WITH: 0
=======================

=======================
STDOUT:
[ 50%] Building C object CMakeFiles/mateExampleCode.elf.dir/notes
[100%] Linking C executable mateExampleCode
[100%] Built target mateExampleCode

=======================

=======================
STDERR:

=======================

And again here is the docker container log mate-executer-1

mate-executor-1            | [2023-07-11 21:10:55,797: INFO/MainProcess] Task mate.tasks.compile.compile_artifact[f233333f-320d-46cf-b3a8-08c1ef3c0426] received
mate-executor-1            | [2023-07-11 21:10:55,798: DEBUG/MainProcess] TaskPool: Apply <function fast_trace_task at 0x7fcb0bad0700> (args:('mate.tasks.compile.compile_artifact', 'f233333f-320d-46cf-b3a8-08c1ef3c0426', {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': 'f233333f-320d-46cf-b3a8-08c1ef3c0426', 'shadow': None, 'eta': None, 'expires': None, 'group': None, 'group_index': None, 'retries': 0, 'timelimit': [None, None], 'root_id': 'f233333f-320d-46cf-b3a8-08c1ef3c0426', 'parent_id': None, 'argsrepr': "('f5b0c6f7cbf541d8800fd2d3078aaef9', CompileOptions(testbed=None, containerized=False, experimental_embed_bitcode=False, docker_image=None, containerized_infer_build=True, make_targets=None, extra_compiler_flags=[]))", 'kwargsrepr': "{'compilation_id': 'd726950941e44049b38cc92283ecba3c'}", 'origin': 'gen28@c312ff637c87', 'ignore_result': False, 'properties': {'content_type': 'application/x-python-serialize', 'content_encoding': 'binary', 'application_headers': {'lang': 'py', 'task': 'mate.tasks.compile.compile_artifact', 'id': 'f233333f-320d-46cf-b3a8-08c1ef3c0426', 'shadow': None, 'eta': None, 'expires': None, 'group':... kwargs:{})
mate-server-1              | INFO:     172.18.0.1:50838 - "POST /api/v1/compilations HTTP/1.1" 201 Created
mate-executor-1            | [2023-07-11 21:10:55,856: DEBUG/ForkPoolWorker-8] http://storage:9000 "GET /artifacts/f5b0c6f7cbf541d8800fd2d3078aaef9 HTTP/1.1" 200 44950
mate-executor-1            | [2023-07-11 21:10:55,899: DEBUG/ForkPoolWorker-8] starting at /tmp/tmpiv9vuzq8/mateCMakeExampleCode; searching [PosixPath('/tmp/tmpiv9vuzq8/mateCMakeExampleCode/CMakeLists.txt'), PosixPath('/tmp/tmpiv9vuzq8/mateCMakeExampleCode/build'), PosixPath('/tmp/tmpiv9vuzq8/mateCMakeExampleCode/notes.c')]
mate-executor-1            | [2023-07-11 21:10:55,899: DEBUG/ForkPoolWorker-8] /tmp/tmpiv9vuzq8/mateCMakeExampleCode does not contain a Makefile; checking the next layer
mate-executor-1            | [2023-07-11 21:10:55,900: DEBUG/ForkPoolWorker-8] found a makefile in /tmp/tmpiv9vuzq8/mateCMakeExampleCode/build
mate-executor-1            | [2023-07-11 21:10:55,901: DEBUG/ForkPoolWorker-8] running args=['blight-exec', '--swizzle-path', '--guess-wrapped', '--', 'make']
mate-executor-1            | [2023-07-11 21:10:56,264: DEBUG/ForkPoolWorker-8] http://storage:9000 "PUT /artifacts/feaace818597482cb159a081af265101 HTTP/1.1" 200 0
mate-executor-1            | [2023-07-11 21:10:56,265: ERROR/ForkPoolWorker-8]
mate-executor-1            | You have encountered a bug in MATE's implementation. Please report it.
mate-executor-1            |
mate-executor-1            | Additional information:
mate-executor-1            | artifact: given an empty fileobj to upload self.uuid='adbb5a5bb64d42cfb47a9eabb12bc137' self.kind=<ArtifactKind.BlightJournal: 'blight:journal'>

@woodruffw
Copy link
Collaborator

Oh well.

Is CMake calling /usr/bin/cc explicitly? That means it isn't honoring CC, which is strange.

@F1o0T
Copy link
Author

F1o0T commented Jul 11, 2023

I have replaced every /usr/bin/cc with $(CC) and it worked!, thanks a lot :)
But I have no idea why CMake is not doing that itself! why is it using /usr/bin/cc instead of $(CC)!

BTW, I have added the suffix .o and .elf back now while having $(CC) instead of /usr/bin/cc, and it worked. But Mate wasn't really telling where exactly the issue untill we went through this process of renaming.

@woodruffw
Copy link
Collaborator

I have replaced every /usr/bin/cc with $(CC) and it worked!, thanks a lot :)

Glad to hear it!

But I have no idea why CMake is not doing that itself! why is it using /usr/bin/cc instead of $(CC)!

Yeah, me neither. It could be an environmental step I missed, some default or assumption that's changed, etc. We unfortunately didn't make this part of MATE super easy to debug 😅

@F1o0T
Copy link
Author

F1o0T commented Jul 11, 2023

Thanks for your time a lot, that was really helpful. Since it is midnight here already; I will run now, and tomorrow I will check why cmake is not generating Makefiles with $(CC). I will keep you posted here!, enjoy your rest of the day :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

5 participants