Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

JUnit 5 Assert Equals Comparison Failure In IntelliJ Is Not Displayed #501

Closed
bensullivan-ag opened this issue Apr 5, 2022 · 8 comments
Closed

JUnit 5 Assert Equals Comparison Failure In IntelliJ Is Not Displayed #501

bensullivan-ag opened this issue Apr 5, 2022 · 8 comments

Comments

@bensullivan-ag
Copy link

bensullivan-ag commented Apr 5, 2022 •
edited
Loading

Hi

I am using AssertJ variant on latest version of JsonUnit in IntelliJ 2021.3.3, Java 11:

dependencies {
    testImplementation 'net.javacrumbs.json-unit:json-unit-assertj:2.33.0'
    testImplementation 'org.junit.jupiter:junit-jupiter:5.8.2'
}

This test with a jsonunit assertion does not display the IntelliJ assert equals comparison failure:

@Test
void jsonUnitShouldProvideIntelliJFailureDiff() {
    assertThatJson("{\"foo\":1, \"bar\":2}").isEqualTo("{\"b\":3, \"a\":1}");
}

Output:

JSON documents are different:
Different keys found in node "", missing: "a","b", extra: "bar","foo", expected: <{"a":1,"b":3}> but was: <{"bar":2,"foo":1}>

org.opentest4j.AssertionFailedError: JSON documents are different:
Different keys found in node "", missing: "a","b", extra: "bar","foo", expected: <{"a":1,"b":3}> but was: <{"bar":2,"foo":1}>

But this junit assertion does:

@Test
void junitShouldProvideIntelliJFailureDiff() {
    assertEquals("{\"foo\":1, \"bar\":2}", "{\"b\":3, \"a\":1}");
}

Output:

expected: <{"foo":1, "bar":2}> but was: <{"b":3, "a":1}>
Expected :{"foo":1, "bar":2}
Actual   :{"b":3, "a":1}
<Click to see difference>

We'd really like to have the comparison failure (available by clicking the <Click to see difference> link in test output) in the IDE as it helps triage the failures with better context for larger json objects.

Is there any way to achieve this?

Ideally it would be great to have both the jsonunit error, which is more descriptive than the junit equivalent, and the link the IntelliJ comparison failure.

Thanks!

Ben
@bensullivan-ag
Copy link
Author

bensullivan-ag commented Apr 5, 2022 •
edited
Loading

This old issue seems to suggest the IntelliJ comparison failure used to work with jsonunit.

@bensullivan-ag bensullivan-ag changed the title JUnit 5 Assert Equals Comparison Failure Is Not Displayed JUnit 5 Assert Equals Comparison Failure In IntelliJ Is Not Displayed Apr 6, 2022
@lukas-krecan
Copy link
Owner

lukas-krecan commented Apr 9, 2022 •
edited
Loading

Hi, thanks for feedback. That's strange, this is how it looks like in my IntelliJ (IntelliJ IDEA 2021.3.2 (Ultimate Edition))

Screen Shot 2022-04-09 at 7 12 27 PM

@lukas-krecan
Copy link
Owner

I am using AssertionFailedError which allows to pass the expected and actual value and the last time I checked the IntelliJ source code, they were using it.

@bensullivan-ag
Copy link
Author

bensullivan-ag commented Apr 21, 2022 •
edited
Loading

That's bizarre...

Here's my full output:

> Task :compileJava NO-SOURCE
> Task :processResources NO-SOURCE
> Task :classes UP-TO-DATE
> Task :compileTestJava
> Task :processTestResources UP-TO-DATE
> Task :testClasses
> Task :test FAILED
Apr. 21, 2022 1:00:34 PM org.junit.platform.launcher.core.EngineDiscoveryOrchestrator lambda$logTestDescriptorExclusionReasons$7
INFO: 0 containers and 4 tests were Method or class mismatch
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.

JSON documents are different:
Different keys found in node "", missing: "a","b", extra: "bar","foo", expected: <{"a":1,"b":3}> but was: <{"bar":2,"foo":1}>

org.opentest4j.AssertionFailedError: JSON documents are different:
Different keys found in node "", missing: "a","b", extra: "bar","foo", expected: <{"a":1,"b":3}> but was: <{"bar":2,"foo":1}>

	at app//net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:35)
	at app//net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45)
	at app//net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:617)
	at app//net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:116)
	at app//api.testing.demo.JsonResponsePayloadDemoTest.jsonUnitShouldProvideIntelliJFailureDiff(JsonResponsePayloadDemoTest.java:62)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
	at app//org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:725)
	at app//org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:140)
	at app//org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestMethod(TimeoutExtension.java:84)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(ExecutableInvoker.java:115)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.lambda$invoke$0(ExecutableInvoker.java:105)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
	at app//org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:104)
	at app//org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:98)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:214)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:210)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:135)
	at app//org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:66)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at [email protected]/java.util.ArrayList.forEach(ArrayList.java:1541)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
	at app//org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
	at app//org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
	at app//org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
	at app//org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
	at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
	at app//org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:108)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
	at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:96)
	at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:75)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:99)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:79)
	at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:75)
	at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at [email protected]/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at [email protected]/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at [email protected]/java.lang.reflect.Method.invoke(Method.java:566)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
	at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
	at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
	at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
	at com.sun.proxy.$Proxy2.stop(Unknown Source)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
	at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
	at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at app//worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)


JsonResponsePayloadDemoTest > jsonUnitShouldProvideIntelliJFailureDiff() FAILED
    org.opentest4j.AssertionFailedError at JsonResponsePayloadDemoTest.java:62
1 test completed, 1 failed
FAILURE: Build failed with an exception.
* What went wrong:
Execution failed for task ':test'.
> There were failing tests. See the report at: file:///Users/besullivan/work/api-testing-demo/build/reports/tests/test/index.html
* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.
* Get more help at https://help.gradle.org
BUILD FAILED in 899ms
3 actionable tasks: 2 executed, 1 up-to-date

Is your stacktrace the same as mine?

What does your gradle build file look like?

Thanks

@lukas-krecan
Copy link
Owner

You can check-out the code and add the test here

@bensullivan-ag
Copy link
Author

I think this might be a gradle nuance.

If you open the gradle project in the fork I've made and run the AbstractAssertJTest test in IntelliJ, what do you see?

https://github.com/bensullivan-ag/JsonUnitIntelliJDiffIssue/tree/master/tests/test-base-gradle

Thanks

Ben

@ljchen1
Copy link

ljchen1 commented Apr 26, 2022

Hi, it should be a issue in the IDEA instead of the JsonUnit.
The related issue is here: https://youtrack.jetbrains.com/issue/IDEA-277216/%3CClick-to-see-difference%3E-is-not-present-in-test-output-when-run
The JsonUnit use the assertj under the hood so it has this issue too.

@lukas-krecan
Copy link
Owner

Makes sense, thanks a lot for investigation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@lukas-krecan @bensullivan-ag @ljchen1