Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ability to customize how objects are printed in case of test failure #7478

Open
4 tasks done
bgoscinski opened this issue Feb 12, 2025 · 2 comments · May be fixed by #7481
Open
4 tasks done

Ability to customize how objects are printed in case of test failure #7478

bgoscinski opened this issue Feb 12, 2025 · 2 comments · May be fixed by #7481
Labels
enhancement: pending triage p2-to-be-discussed Enhancement under consideration (priority)

Comments

@bgoscinski
Copy link
Contributor

bgoscinski commented Feb 12, 2025

Clear and concise description of the problem

Currently, when a test fails, Vitest attempts to print the diff if the objects are of the same type or both received and expected values, which is very helpful when the diff is short and doesn't show unnecessary implementation details. However, when an object that references a complex object graph is being printed, Vitest takes a very long time and consumes a lot of memory, which sometimes causes OOM errors.

This problem is demonstrated in this stackblitz where I'm using Vue to simulate a simple reactivity graph without cycles and try to assert that the final computed instance equals to { this_will_fail: true }. Test correctly fails, but the diff printed in the console is HUGE and unhelpful.

When the computed chain gets longer (I tried with CHAIN_LENGTH = 100) then Vitest never completes and the browser tab consumes about 4 GB of memory (at which point I run out of patience).

Suggested solution

I would like to be able to customize how Vitest prints certain objects I'm using in my code, such as Vue's computed and refs, to omit everything except the value property. I'm envisionning an API similar to expect.addSnapshotSerializer where I can provide a test function and some kind of printer/replacer/mapper that would allow me to customize the printing of objects that pass the test function. From that printer/replacer/mapper function it's important to be able to defer to Vitest's default behavior for nested properties. For example, for Vue refs, I'd like to print something like:

  Ref (${defaultVitestPrinter(myRef.value)})

Alternative

@hi-ogawa in #7092 suggested monkey patching prototype of Vue reactivity primitives by adding toJSON implementation like this. There are two problems with this approach:

  1. It works only when Vitest is diffing an object against another object but doesn't seem to work when comparing to a value of a different type, as shown here
  2. I'm hesitant to monkey patch theprototypes of refs in Vue because I'm afraid that this may have unintended and hard-to-predict consequences in tests

Additional context

I think this is different in scope from #7429 because this feature request is intended to minimize diff output of certain types of object, while #7429 is about the time complexity of the toEqual check. In my case, toEqual returns quickly, but the diff printing is the problem.

Maybe this could help in #7461

Validations

@bgoscinski bgoscinski changed the title Customize Object Serialization for Test Failure Output Ability to customize how objects are printed in case of test failure Feb 12, 2025
@hi-ogawa
Copy link
Contributor

hi-ogawa commented Feb 13, 2025

I made a self-contained repro https://stackblitz.com/edit/vitest-dev-vitest-kbpcsw76?file=test%2Frepro.test.ts

Interestingly, Node's assert.deepStrictEqual diff also doesn't seem to be robust against this, and it causes OOM immediately even with smaller size of cyclic structure. This reproduces only since Node 22, which landed a new assert diff algorithm nodejs/node#54862 as the old diff seems to bail out early and truncate the output.

@hi-ogawa
Copy link
Contributor

hi-ogawa commented Feb 13, 2025

I'm thinking about a following change to improve the situation:

  • set default maxDepth to be not Infinity to reduce OOM crashes in worst case by default
  • add more diff options https://vitest.dev/config/#diff to allow customize for users' need
    • it can start from maxDepth, but maybe we can allow passing pretty-format plugins

@hi-ogawa hi-ogawa added the p2-to-be-discussed Enhancement under consideration (priority) label Feb 13, 2025
@hi-ogawa hi-ogawa moved this to P2 - 2 in Team Board Feb 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement: pending triage p2-to-be-discussed Enhancement under consideration (priority)
Projects
Status: P2 - 2
2 participants