Test failure with Matplotlib 3.9.0rc2

[QuLogic]

Currently, only one test fails; TestComparisons.test_Surface:

_________________________ TestComparisons.test_Surface _________________________

args = (<tests.test_blocks.TestComparisons object at 0x7f59b179b710>,)
kwargs = {}, anim = <animatplot.animation.Animation object at 0x7f59b1592cf0>
fignum = 1, fig = <Figure size 640x480 with 1 Axes>

    @functools.wraps(func)
    def wrapper(*args, **kwargs):
        # First close anything from previous tests
        plt.close("all")
    
        anim = func(*args, **kwargs)
        if remove_text:
            fignum = plt.get_fignums()[0]
            fig = plt.figure(fignum)
            remove_ticks_and_titles(fig)
        try:
>           _compare_animation(anim, baseline_images, fmt, nframes, tol)

tests/tools.py:86: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

anim = <animatplot.animation.Animation object at 0x7f59b1592cf0>
expected = 'Blocks/Surface', format_ = '.png', nframes = 3, tol = 0.001

    def _compare_animation(anim, expected, format_, nframes, tol):
        # generate images from the animation
        base_dir, filename = split(join("tests", "baseline_images", expected))
        out_dir = split(join("tests", "output_images", expected))[0]
    
        if not os.path.exists(out_dir):
            os.makedirs(out_dir)
    
        anim.save(os.path.join(out_dir, (filename + format_)), writer=BunchOFiles())
    
        for i in range(nframes):
            image_name = "%s%d%s" % (filename, i, format_)
            expected_name = os.path.join(base_dir, image_name)
            actual_name = os.path.join(out_dir, image_name)
    
            err = compare_images(expected_name, actual_name, tol, in_decorator=True)
    
            if not os.path.exists(expected_name):
                raise ImageComparisonFailure("image does not exist: %s" % expected_name)
    
            if err:
                import base64
    
                print("import base64")
                for key in ["actual", "expected"]:
                    err[key] = os.path.relpath(err[key])
                    with open(err[key], "rb") as f:
                        print(
                            f"with open(\"{err[key].replace('/','_')}\", \"wb\") as f:\n    f.write(base64.b64decode({base64.b64encode(f.read())}))\n"
                        )
    
>               raise ImageComparisonFailure(
                    "images not close (RMS %(rms).3f):\n\t%(actual)s\n\t%(expected)s " % err
                )
E               matplotlib.testing.exceptions.ImageComparisonFailure: images not close (RMS 9.279):
E               	tests/output_images/Blocks/Surface0.png
E               	tests/baseline_images/Blocks/Surface0.png

I think some of it might be related to matplotlib/matplotlib#25272 but also, this test seems far to complex for the 3D renderer in Matplotlib (which is not as good as a real 3D renderer.) Since this was also a problem in #13 for 3.8.0rc1, it might be a good idea to simplifiy the dataset for this test.

[dschwoerer]

Yes, it probably would be good to simplify the test. However, I am not sure just changing the data would have helped in this case.

For now, I think I will just keep updating the images for new matplotlib versions :-/