Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Flaky test #326

Open
rossta opened this issue Dec 18, 2024 · 0 comments
Open

Flaky test #326

rossta opened this issue Dec 18, 2024 · 0 comments

Comments

@rossta
Copy link
Contributor

rossta commented Dec 18, 2024

Describe the bug

When I run rspec with a specific seed, the suite fails on the page_embedding_spec.rb:

$ bin/rspec --seed 52318 --fail-fast

Randomized with seed 52318
.........................................................................................................................................F

Failures:

  1) PageEmbedding serializes and deserializes embedding vector
     Failure/Error: ActiveSupport::JSON.decode(value)
     
     JSON::ParserError:
       unexpected token at '_?;?)?????p'
     # /.gem/ruby/3.3.5/gems/json-2.9.0/lib/json/common.rb:221:in `parse'
     # /.gem/ruby/3.3.5/gems/json-2.9.0/lib/json/common.rb:221:in `parse'
     # ./app/models/types/vector.rb:16:in `deserialize'
     # ./spec/models/page_embedding_spec.rb:18:in `block (2 levels) in <top (required)>'
     # ./spec/support/webmock_vcr.rb:40:in `block (3 levels) in <top (required)>'
     # /.gem/ruby/3.3.5/gems/vcr-6.3.1/lib/vcr.rb:274:in `turned_off'
     # ./spec/support/webmock_vcr.rb:40:in `block (2 levels) in <top (required)>'
     # /.gem/ruby/3.3.5/gems/webmock-3.24.0/lib/webmock/rspec.rb:39:in `block (2 levels) in <top (required)>'

Finished in 11.12 seconds (files took 3.88 seconds to load)
138 examples, 1 failure

Failed examples:

rspec ./spec/models/page_embedding_spec.rb:8 # PageEmbedding serializes and deserializes embedding vector

Randomized with seed 52318

I can reproduce locally. This has happened at least once in CI: https://github.com/joyofrails/joyofrails.com/actions/runs/12402409885/job/34623858417

The test typically passed otherwise.

The :random_embedding factory most certainly plays a role.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant