-
Notifications
You must be signed in to change notification settings - Fork 225
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
why are there missing files #820
Comments
Hi,
Do you mean that entries in "Currently analyzed firmware" (/system_health) never complete? That could happen if there are errors during analysis or unpacking and the file gets lost during scheduling (but it should obviously not happen). Since there don't seem to be any helpful log messages, it could be complicated to debug the problem. Did you maybe see any errors or stack traces in the terminal output ( |
Correct. I rerun the test and reproduced the error, but I now see that there is a time out exception that seems to be not handled correctly, maybe because when handling an exception other exceptions were raised. Attached is the exception log message. The extractor that seems to take a long time before the exception is binwalk as seen by ps. |
The error is indeed not handled correctly. Nevertheless, it is also not clear what caused the error in the first place. Was it a particularly large or in some other way unusual file? Running binwalk usually takes some time for large files (which may be the cause of the timeout). |
The issue here I think is that binwalk does indeed take too much time for some files, and that FACT_core does not correctly handle timeouts in FACT_extractor. In some cases binwalk extracts bogus data and as FACT_extractor is called in a recursive manner, a very large file can be sent to binwalk for further extraction. Feel free to assign this to me. |
We are always happy to receive external contributions and will try to support you, so feel free to try to improve this. Some things to note:
|
…d#820 Timeouts are handled by catching all requests exceptions because requests raise ConnectionError on a timeout.
I submited a merge request to fix any timeout in fact_extractor. I tested this on v3.3. Unfortunately I could not test this on main, but I think it should still work. |
I also submited a patch to fact_extractor to try to get partial results in case binwalk does not finish. |
I think it's worth to mention the PR: #852 |
And the other PR is fkie-cad/fact_extractor#94 |
Hello! First of all: thank you guys so much for the contributions here. Unfortunately, our lead developer @jstucke and his right hand @maringuu are pretty busy this week, which is why probably nothing will happen until the 19th. Just giving you a heads up - normally both PRs would've already been considered :-) |
Handle timeout errors when calling fact_extractor, related to #820
Since this should hopefully fixed with #852 and fkie-cad/fact_extractor#94, I will close this issue. Feel free to reopen if this is still a thing. |
For some firmware images, the analysis never finishes, and there are missing files in the Admin/Find Missing Analysis tab.
Is this expected? In case no, how can I see what it went wrong to fix it? There are no messages on the Logs tab, and I am using FACT_docker.
The text was updated successfully, but these errors were encountered: