Diff ouptut can be confusing when assertEquals fails on comparing objects or arrays containing \DateTime

[devwebpeanuts]

Q	A
PHPUnit version	11.4.1
PHP version	8.3.9
Installation Method	Composer

Summary

In some cases, the diff ouptut can be confusing when assertEquals fails on comparing objects or arrays containing \DateTime.

Current behavior

Given this test

public function testMisleadingDiffOutput()
{
    $baz = new \StdClass();
    $baz->foobar = "A";

    $qux = new \StdClass();
    $qux->foobar = "B";

    $today = new \DateTimeImmutable("today");
    $yesterday = new \DateTimeImmutable("yesterday");

    $expected = [
        [$baz, $today],
        [$qux, $yesterday],
    ];
    $actual = [
        [$qux, $yesterday],
        [$baz, $today],
    ];

    Assert::assertEquals($expected, $actual);
}

Testing output below is generated

PHPUnit 11.4.1 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.9
Configuration: /var/www/app/phpunit.xml

F                                                                   1 / 1 (100%)

Time: 00:00.005, Memory: 8.00 MB

There was 1 failure:

1) DumbTest::testMisleadingDiffOutput
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (
         0 => stdClass Object (
-            'foobar' => 'A'
+            'foobar' => 'B'
         )
-        1 => 2024-10-17T00:00:00.000000+0000
+        1 => 2024-10-16T00:00:00.000000+0000
     )
     1 => Array (
         0 => stdClass Object (...)
-        1 => 2024-10-16T00:00:00.000000+0000
+        1 => 2024-10-17T00:00:00.000000+0000
     )
 )

It's confusing because as diff is given for \DateTime objects, it may suggest that stdClass instances in second element of actual and expected array are equals.

Expected behavior

After some investigation, I saw that ObjectComparator keeps track of processed pair of expected and actual values to avoid redundant comparison: that's why diff on stdClass is not displayed on second elements.
DateTimeComparator, even though it extends ObjectComparator does not have the same behavior on keeping track of processed comparisons. I think it should.

As a result, the testing output could be

PHPUnit 11.4.1 by Sebastian Bergmann and contributors.

Runtime:       PHP 8.3.9
Configuration: /var/www/app/phpunit.xml

F                                                                   1 / 1 (100%)

Time: 00:00.005, Memory: 8.00 MB

There was 1 failure:

1) DumbTest::testMisleadingDiffOutput
Failed asserting that two arrays are equal.
--- Expected
+++ Actual
@@ @@
 Array (
     0 => Array (
         0 => stdClass Object (
-            'foobar' => 'A'
+            'foobar' => 'B'
         )
-        1 => 2024-10-17T00:00:00.000000+0000
+        1 => 2024-10-16T00:00:00.000000+0000
     )
     1 => [...]
 )

Additional information

phpunit.xml

<?xml version="1.0" encoding="UTF-8"?>
<phpunit xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:noNamespaceSchemaLocation="https://schema.phpunit.de/11.4/phpunit.xsd"
         bootstrap="vendor/autoload.php"
         cacheDirectory=".phpunit.cache"
         colors="true"
         executionOrder="depends,defects"
         requireCoverageMetadata="false"
         beStrictAboutCoverageMetadata="true"
         beStrictAboutOutputDuringTests="true"
         failOnWarning="true"
         failOnRisky="true"
         failOnDeprecation="true"
         failOnNotice="true"
         failOnSkipped="true"
         failOnIncomplete="true">
    <testsuites>
        <testsuite name="default">
            <directory>tests/</directory>
        </testsuite>
    </testsuites>

    <source restrictDeprecations="true" restrictNotices="true" restrictWarnings="true">
        <include>
            <directory>src</directory>
        </include>
    </source>
</phpunit>

composer.json

{
  "require": {
    "php": "^8.3"
  },
  "require-dev": {
    "phpunit/phpunit": "^11.4"
  }
}

composer info | sort

myclabs/deep-copy                  1.12.0 Create deep copies (clones) of yo...
nikic/php-parser                   v5.3.1 A PHP parser written in PHP
phar-io/manifest                   2.0.4  Component for reading phar.io man...
phar-io/version                    3.2.1  Library for handling version info...
sebastian/cli-parser               3.0.2  Library for parsing CLI options
sebastian/code-unit-reverse-lookup 4.0.1  Looks up which function or method...
sebastian/code-unit                3.0.1  Collection of value objects that ...
sebastian/comparator               6.1.0  Provides the functionality to com...
sebastian/complexity               4.0.1  Library for calculating the compl...
sebastian/diff                     6.0.2  Diff implementation
sebastian/environment              7.2.0  Provides functionality to handle ...
sebastian/exporter                 6.1.3  Provides the functionality to exp...
sebastian/global-state             7.0.2  Snapshotting of global state
sebastian/lines-of-code            3.0.1  Library for counting the lines of...
sebastian/object-enumerator        6.0.1  Traverses array structures and ob...
sebastian/object-reflector         4.0.1  Allows reflection of object attri...
phpunit/php-code-coverage          11.0.7 Library that provides collection,...
phpunit/php-file-iterator          5.1.0  FilterIterator implementation tha...
phpunit/php-invoker                5.0.1  Invoke callables with a timeout
phpunit/php-text-template          4.0.1  Simple template engine.
phpunit/php-timer                  7.0.1  Utility class for timing
phpunit/phpunit                    11.4.1 The PHP Unit Testing framework.
sebastian/recursion-context        6.0.2  Provides functionality to recursi...
sebastian/type                     5.1.0  Collection of value objects that ...
sebastian/version                  5.0.2  Library that helps with managing ...
theseer/tokenizer                  1.2.3  A small library for converting to...