Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error in instrumented process when Fuzzing a recursive class #2545

Open
alisevych opened this issue Aug 22, 2023 · 3 comments
Open

Error in instrumented process when Fuzzing a recursive class #2545

alisevych opened this issue Aug 22, 2023 · 3 comments
Assignees
Labels
comp-fuzzing Issue is related to the fuzzing comp-instrumented-process Issue is related to Instrumented process ctg-bug Issue is a bug

Comments

@alisevych
Copy link
Member

alisevych commented Aug 22, 2023

Description

Error in instrumented process when Fuzzing IR method in Unit tests

To Reproduce

  1. Install UnitTestBot plugin built from main in IntelliJ IDEA
  2. Open UTBotJava project
  3. Set Fuzzing 100% mode //not strictly necessary
  4. Generate tests for Recursion class in utbot-sample

Expected behavior

No errors in instrumented process are expected.
Class is expected to be correctly instantiated and methods executed.

Actual behavior

There is error report with 11K-20K occurrences of Default concrete execution failed
for every method under test

Screenshots, logs

public final class RecursionTest {
    ///region Test suites for executable org.utbot.examples.recursion.Recursion.fib

    ///region Errors report for fib

    public void testFib_errors() {
        // Couldn't generate some tests. List of errors:
        // 
        // 11099 occurrences of:
        // Default concrete execution failed

    }
    ///endregion

    ///endregion

Environment

IntelliJ IDEA version - 2023.2 Ultimate
Project - Maven/Gradle
JDK - 1.8/17

@alisevych alisevych added ctg-bug Issue is a bug spec-internal Internal issue comp-instrumented-process Issue is related to Instrumented process comp-spring Issue is related to Spring projects support labels Aug 22, 2023
@alisevych alisevych added this to the Spring Phase 4 milestone Aug 22, 2023
@alisevych alisevych changed the title Error in instrumented process when Fuzzing IR method in Unit tests Error in instrumented process when Fuzzing Aug 22, 2023
@alisevych alisevych added the comp-fuzzing Issue is related to the fuzzing label Aug 23, 2023
@alisevych alisevych changed the title Error in instrumented process when Fuzzing Error in instrumented process when Fuzzing a recursive class Aug 23, 2023
@alisevych alisevych removed the spec-internal Internal issue label Aug 23, 2023
@IlyaMuravjov IlyaMuravjov removed the comp-spring Issue is related to Spring projects support label Aug 24, 2023
@IlyaMuravjov
Copy link
Collaborator

IlyaMuravjov commented Aug 24, 2023

After setting Hanging test timeout to 100 ms, enabling Power saving mode on my laptop (to worsen the performance) and generating tests a few times, I've managed to get some concrete execution failures.

Here are the logs: utbot-engine-current.log.

It seems that concrete failures were caused by timeout happening during evaluation of static initialization block in JavaStdLibCustomModelConstructorsKt, which caused class loader to mark JavaStdLibCustomModelConstructorsKt as non-loadable, since it can't be initialized, which in turn made all later invocations to fail with NoClassDefFoundError: Could not initialize class JavaStdLibCustomModelConstructorsKt.

That raises the question about of safe termination of hanging threads. I propose to do class loading, executing <clinit> methods, and thread termination under same monitor to avoid such problems, however further discussion is advised.

@alisevych
Copy link
Member Author

@tyuldashev
The issue is not reproducing for me in UTBotJava with plugin built from latest main.
Can you please verify it?

@alisevych alisevych removed this from the Spring Phase 4 milestone Sep 19, 2023
@tyuldashev
Copy link
Collaborator

Yep, not reproducible at the moment. Though if probable cause is identified then we maybe should not close case without further discussion.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
comp-fuzzing Issue is related to the fuzzing comp-instrumented-process Issue is related to Instrumented process ctg-bug Issue is a bug
Projects
Status: Todo
Development

No branches or pull requests

4 participants