Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix fetching search results and handle fetching errors #2618

Merged
merged 7 commits into from
Jul 28, 2023

Conversation

obulat
Copy link
Contributor

@obulat obulat commented Jul 11, 2023

Fixes

Fixes #2588 by @obulat

Description

This PR moves the fetching of search results from the asyncData to the search middleware and the useFetch hook in the search.vue page.

Fetching search results on the server

Just like in the single result pages, on the server, we make sure that we can fetch the data or redirect to the Nuxt error page. In the search.vue page, we set fetchOnServer to false so that the data that was fetched on the server (if the search page was opened directly, when you go to https://openverse.org/search/?q=cat) isn't refetched when the page is hydrating.

Fetching search results on the client

If we navigate to the search page from other pages (homepage or 404 page), the useFetchcalls fetchMedia in search.vue to get the search results. If the route changes (either the path, i.e. when the search type changes, or the query, i.e., when the search term, or filters change), the router watch calls fetchMedia in search.vue.

Updating the provider filters

Previously, asyncData updated the search state and provider filters. Here, this is moved to the search middleware.

Fetching errors

When there are no search results for a query, we receive a 200 response with an empty result list. In the media store, we set fetchingError to 404 status in case there are no results, to show the 404 page. fetchingError.message to NO_RESULT to show the No results page.

All content errors

This PR updates the handling of the "All content" errors: if there is even a single 429 error, we set the combined allMediaError status to 429, because it means that all client searches will be throttled.
allMediaError is set to 404 status only if the fetchingErrors for all media types is 404, so we only show a 404 page for "All content" if both audio and image had no results. After @dhruvkb's request for change, I've updated allMediaError to set a NO_RESULT error message if there are no results for all supported media types. Otherwise, we show the available media type results.

Testing Instructions

Searching from the homepage and 404 page, as well as from the search page's header should work correctly.
Clicking "Load more" should load the next page of the results for all search types.

For all media, check cases with only one type of results, such as "ecommerce image".
Also, try queries that don't have any results (such as "querywithnoresults").

Checklist

  • My pull request has a descriptive title (not a vague title likeUpdate index.md).
  • My pull request targets the default branch of the repository (main) or a parent feature branch.
  • My commit messages follow best practices.
  • My code follows the established code style of the repository.
  • I added or updated tests for the changes I made (if applicable).
  • I added or updated documentation (if applicable).
  • I tried running the project locally and verified that there are no visible errors.
  • I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.


Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

@github-actions github-actions bot added the 🧱 stack: frontend Related to the Nuxt frontend label Jul 11, 2023
@openverse-bot openverse-bot added 🟧 priority: high Stalls work on the project or its dependents ✨ goal: improvement Improvement to an existing user-facing feature 💻 aspect: code Concerns the software code in the repository labels Jul 11, 2023
@github-actions
Copy link

github-actions bot commented Jul 11, 2023

Size Change: +506 B (0%)

Total Size: 839 kB

Filename Size Change
./frontend/.nuxt/dist/client/app.js 123 kB +256 B (0%)
./frontend/.nuxt/dist/client/app.modern.js 114 kB +185 B (0%)
./frontend/.nuxt/dist/client/commons/app.modern.js 80.8 kB +22 B (0%)
./frontend/.nuxt/dist/client/pages/search.js 2.27 kB +11 B (0%)
./frontend/.nuxt/dist/client/pages/search.modern.js 2.09 kB +34 B (+2%)
./frontend/.nuxt/dist/client/vendors/app.js 68.3 kB -10 B (0%)
./frontend/.nuxt/dist/client/vendors/app.modern.js 68.1 kB -12 B (0%)
ℹ️ View Unchanged
Filename Size Change
./frontend/.nuxt/dist/client/234.js 273 B 0 B
./frontend/.nuxt/dist/client/234.modern.js 277 B 0 B
./frontend/.nuxt/dist/client/235.js 1.85 kB 0 B
./frontend/.nuxt/dist/client/commons/app.js 90.2 kB -3 B (0%)
./frontend/.nuxt/dist/client/commons/components/v-search-grid/pages/search.js 5.17 kB 0 B
./frontend/.nuxt/dist/client/commons/components/v-search-grid/pages/search.modern.js 5.61 kB 0 B
./frontend/.nuxt/dist/client/components/loading-icon.js 731 B -1 B (0%)
./frontend/.nuxt/dist/client/components/loading-icon.modern.js 733 B 0 B
./frontend/.nuxt/dist/client/components/table-sort-icon.js 514 B 0 B
./frontend/.nuxt/dist/client/components/table-sort-icon.modern.js 519 B 0 B
./frontend/.nuxt/dist/client/components/v-all-results-grid.js 4.03 kB -3 B (0%)
./frontend/.nuxt/dist/client/components/v-all-results-grid.modern.js 3.98 kB 0 B
./frontend/.nuxt/dist/client/components/v-all-results-grid/pages/search/audio/pages/search/index.js 3.19 kB +4 B (0%)
./frontend/.nuxt/dist/client/components/v-all-results-grid/pages/search/audio/pages/search/index.modern.js 3.08 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-cell.js 570 B -1 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-cell.modern.js 576 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-details.js 1.79 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-details.modern.js 1.77 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-track-skeleton.js 958 B 0 B
./frontend/.nuxt/dist/client/components/v-audio-track-skeleton.modern.js 961 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-track.js 5.39 kB +7 B (0%)
./frontend/.nuxt/dist/client/components/v-audio-track.modern.js 5.35 kB +5 B (0%)
./frontend/.nuxt/dist/client/components/v-back-to-search-results-link.js 635 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-back-to-search-results-link.modern.js 641 B 0 B
./frontend/.nuxt/dist/client/components/v-bone.js 632 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-bone.modern.js 636 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-box-layout.js 1.33 kB 0 B
./frontend/.nuxt/dist/client/components/v-box-layout.modern.js 1.33 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-content-link.js 1.06 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-content-link.modern.js 1.06 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-content-page.js 531 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-content-page.modern.js 535 B -1 B (0%)
./frontend/.nuxt/dist/client/components/v-content-report-button.js 493 B 0 B
./frontend/.nuxt/dist/client/components/v-content-report-button.modern.js 497 B 0 B
./frontend/.nuxt/dist/client/components/v-content-report-form.js 3.33 kB +2 B (0%)
./frontend/.nuxt/dist/client/components/v-content-report-form.modern.js 3.21 kB 0 B
./frontend/.nuxt/dist/client/components/v-content-report-popover.js 3.8 kB +3 B (0%)
./frontend/.nuxt/dist/client/components/v-content-report-popover.modern.js 3.67 kB -2 B (0%)
./frontend/.nuxt/dist/client/components/v-copy-button.js 3.99 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-copy-button.modern.js 4 kB 0 B
./frontend/.nuxt/dist/client/components/v-copy-license.js 2.28 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-copy-license.modern.js 2.25 kB -3 B (0%)
./frontend/.nuxt/dist/client/components/v-dmca-notice.js 793 B 0 B
./frontend/.nuxt/dist/client/components/v-dmca-notice.modern.js 797 B -1 B (0%)
./frontend/.nuxt/dist/client/components/v-error-image.js 2.42 kB +2 B (0%)
./frontend/.nuxt/dist/client/components/v-error-image.modern.js 2.38 kB 0 B
./frontend/.nuxt/dist/client/components/v-error-section.js 372 B 0 B
./frontend/.nuxt/dist/client/components/v-error-section.modern.js 377 B 0 B
./frontend/.nuxt/dist/client/components/v-external-search-form.js 2 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-external-search-form.modern.js 1.98 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-external-source-list.js 897 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-external-source-list.modern.js 896 B 0 B
./frontend/.nuxt/dist/client/components/v-full-layout.js 1.57 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-full-layout.modern.js 1.58 kB +2 B (0%)
./frontend/.nuxt/dist/client/components/v-grid-skeleton.js 1.53 kB +3 B (0%)
./frontend/.nuxt/dist/client/components/v-grid-skeleton.modern.js 1.53 kB +2 B (0%)
./frontend/.nuxt/dist/client/components/v-home-gallery.js 4.28 kB 0 B
./frontend/.nuxt/dist/client/components/v-home-gallery.modern.js 4.26 kB 0 B
./frontend/.nuxt/dist/client/components/v-homepage-content.js 1.8 kB 0 B
./frontend/.nuxt/dist/client/components/v-homepage-content.modern.js 1.77 kB 0 B
./frontend/.nuxt/dist/client/components/v-icon-button.js 523 B 0 B
./frontend/.nuxt/dist/client/components/v-icon-button.modern.js 528 B 0 B
./frontend/.nuxt/dist/client/components/v-image-cell.js 1.95 kB -3 B (0%)
./frontend/.nuxt/dist/client/components/v-image-cell.modern.js 1.94 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-image-details.js 1.44 kB -3 B (0%)
./frontend/.nuxt/dist/client/components/v-image-details.modern.js 1.43 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-image-grid.js 4.42 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-image-grid.modern.js 4.29 kB +5 B (0%)
./frontend/.nuxt/dist/client/components/v-license-tab-panel.js 641 B -1 B (0%)
./frontend/.nuxt/dist/client/components/v-license-tab-panel.modern.js 649 B -1 B (0%)
./frontend/.nuxt/dist/client/components/v-load-more.js 1.18 kB 0 B
./frontend/.nuxt/dist/client/components/v-load-more.modern.js 1.06 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-media-license.js 932 B 0 B
./frontend/.nuxt/dist/client/components/v-media-license.modern.js 939 B 0 B
./frontend/.nuxt/dist/client/components/v-media-reuse.js 2.94 kB +2 B (0%)
./frontend/.nuxt/dist/client/components/v-media-reuse.modern.js 2.91 kB -2 B (0%)
./frontend/.nuxt/dist/client/components/v-media-tag.js 416 B 0 B
./frontend/.nuxt/dist/client/components/v-media-tag.modern.js 419 B 0 B
./frontend/.nuxt/dist/client/components/v-modal.js 1.01 kB 0 B
./frontend/.nuxt/dist/client/components/v-modal.modern.js 997 B 0 B
./frontend/.nuxt/dist/client/components/v-no-results.js 855 B 0 B
./frontend/.nuxt/dist/client/components/v-no-results.modern.js 858 B 0 B
./frontend/.nuxt/dist/client/components/v-radio.js 1 kB -2 B (0%)
./frontend/.nuxt/dist/client/components/v-radio.modern.js 1.01 kB -3 B (0%)
./frontend/.nuxt/dist/client/components/v-related-audio.js 1.73 kB -2 B (0%)
./frontend/.nuxt/dist/client/components/v-related-audio.modern.js 1.64 kB +3 B (0%)
./frontend/.nuxt/dist/client/components/v-related-images.js 805 B -2 B (0%)
./frontend/.nuxt/dist/client/components/v-related-images.modern.js 719 B 0 B
./frontend/.nuxt/dist/client/components/v-report-desc-form.js 975 B -1 B (0%)
./frontend/.nuxt/dist/client/components/v-report-desc-form.modern.js 980 B 0 B
./frontend/.nuxt/dist/client/components/v-row-layout.js 1.72 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-row-layout.modern.js 1.73 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-scroll-button.js 891 B 0 B
./frontend/.nuxt/dist/client/components/v-scroll-button.modern.js 891 B 0 B
./frontend/.nuxt/dist/client/components/v-search-grid.js 6.83 kB +6 B (0%)
./frontend/.nuxt/dist/client/components/v-search-grid.modern.js 6.18 kB 0 B
./frontend/.nuxt/dist/client/components/v-search-results-title.js 618 B 0 B
./frontend/.nuxt/dist/client/components/v-search-results-title.modern.js 622 B 0 B
./frontend/.nuxt/dist/client/components/v-server-timeout.js 298 B 0 B
./frontend/.nuxt/dist/client/components/v-server-timeout.modern.js 303 B 0 B
./frontend/.nuxt/dist/client/components/v-sketch-fab-viewer.js 1.01 kB -1 B (0%)
./frontend/.nuxt/dist/client/components/v-sketch-fab-viewer.modern.js 916 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-snackbar.js 1.07 kB +1 B (0%)
./frontend/.nuxt/dist/client/components/v-snackbar.modern.js 1.07 kB -2 B (0%)
./frontend/.nuxt/dist/client/components/v-sources-table.js 14.4 kB +3 B (0%)
./frontend/.nuxt/dist/client/components/v-sources-table.modern.js 14.4 kB -3 B (0%)
./frontend/.nuxt/dist/client/components/v-warning-suppressor.js 307 B +1 B (0%)
./frontend/.nuxt/dist/client/components/v-warning-suppressor.modern.js 311 B 0 B
./frontend/.nuxt/dist/client/pages/about.js 1.42 kB 0 B
./frontend/.nuxt/dist/client/pages/about.modern.js 1.42 kB 0 B
./frontend/.nuxt/dist/client/pages/audio/_id/index.js 6.73 kB +3 B (0%)
./frontend/.nuxt/dist/client/pages/audio/_id/index.modern.js 6.57 kB -7 B (0%)
./frontend/.nuxt/dist/client/pages/feedback.js 1.35 kB 0 B
./frontend/.nuxt/dist/client/pages/feedback.modern.js 1.36 kB 0 B
./frontend/.nuxt/dist/client/pages/image/_id/index.js 7.05 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/image/_id/index.modern.js 6.87 kB -1 B (0%)
./frontend/.nuxt/dist/client/pages/image/_id/report.js 4.89 kB -1 B (0%)
./frontend/.nuxt/dist/client/pages/image/_id/report.modern.js 4.67 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/index.js 6.34 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/index.modern.js 6.3 kB -1 B (0%)
./frontend/.nuxt/dist/client/pages/preferences.js 1.32 kB 0 B
./frontend/.nuxt/dist/client/pages/preferences.modern.js 1.31 kB -1 B (0%)
./frontend/.nuxt/dist/client/pages/privacy.js 1.26 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/privacy.modern.js 1.26 kB 0 B
./frontend/.nuxt/dist/client/pages/search-help.js 1.62 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/search-help.modern.js 1.61 kB -1 B (0%)
./frontend/.nuxt/dist/client/pages/search/audio.js 1.41 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/search/audio.modern.js 1.39 kB -1 B (0%)
./frontend/.nuxt/dist/client/pages/search/image.js 544 B 0 B
./frontend/.nuxt/dist/client/pages/search/image.modern.js 548 B -1 B (0%)
./frontend/.nuxt/dist/client/pages/search/index.js 442 B 0 B
./frontend/.nuxt/dist/client/pages/search/index.modern.js 4.13 kB +2 B (0%)
./frontend/.nuxt/dist/client/pages/search/model-3d.js 243 B 0 B
./frontend/.nuxt/dist/client/pages/search/model-3d.modern.js 246 B 0 B
./frontend/.nuxt/dist/client/pages/search/search-page.types.js 266 B 0 B
./frontend/.nuxt/dist/client/pages/search/search-page.types.modern.js 271 B +1 B (0%)
./frontend/.nuxt/dist/client/pages/search/video.js 240 B 0 B
./frontend/.nuxt/dist/client/pages/search/video.modern.js 243 B 0 B
./frontend/.nuxt/dist/client/pages/sources.js 1.53 kB +1 B (0%)
./frontend/.nuxt/dist/client/pages/sources.modern.js 1.54 kB 0 B
./frontend/.nuxt/dist/client/runtime.js 2.72 kB 0 B
./frontend/.nuxt/dist/client/runtime.modern.js 2.72 kB 0 B

compressed-size-action

@obulat obulat force-pushed the fix/media_error_handling branch 7 times, most recently from 672cbb3 to 99ab42e Compare July 15, 2023 04:36
@obulat obulat changed the base branch from main to fix/playwright_error_tests July 17, 2023 16:51
@obulat obulat self-assigned this Jul 17, 2023
@obulat obulat changed the title Fix/media error handling Fix fetching search results and handle fetching errors Jul 17, 2023
Base automatically changed from fix/playwright_error_tests to main July 18, 2023 03:30
@obulat obulat force-pushed the fix/media_error_handling branch 4 times, most recently from d8a1a0b to 9915ead Compare July 18, 2023 12:07
@obulat obulat changed the base branch from main to fix/external_sources_test July 18, 2023 12:08
@obulat obulat changed the base branch from fix/external_sources_test to main July 18, 2023 14:11
@obulat obulat changed the base branch from main to fix/external_sources_test July 18, 2023 14:11
@obulat obulat force-pushed the fix/media_error_handling branch 2 times, most recently from 30f314e to 82e52b6 Compare July 18, 2023 14:25
@obulat obulat marked this pull request as ready for review July 18, 2023 14:55
@obulat obulat force-pushed the fix/media_error_handling branch 2 times, most recently from 99af1d3 to 5862fda Compare July 20, 2023 06:15
Copy link
Collaborator

@AetherUnbound AetherUnbound left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes generally make sense to me! I followed the testing instructions and everything worked as expected.

Comment on lines +123 to +130
/**
* Search middleware runs when the path changes. This watcher
* is necessary to handle the query changes.
*
* It updates the search store state from the URL query,
* fetches media, and scrolls to top if necessary.
*/
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for adding these comments!

@openverse-bot
Copy link
Collaborator

Based on the high urgency of this PR, the following reviewers are being gently reminded to review this PR:

@dhruvkb
This reminder is being automatically generated due to the urgency configuration.

Excluding weekend1 days, this PR was ready for review 3 day(s) ago. PRs labelled with high urgency are expected to be reviewed within 2 weekday(s)2.

@obulat, if this PR is not ready for a review, please draft it to prevent reviewers from getting further unnecessary pings.

Footnotes

  1. Specifically, Saturday and Sunday.

  2. For the purpose of these reminders we treat Monday - Friday as weekdays. Please note that the operation that generates these reminders runs at midnight UTC on Monday - Friday. This means that depending on your timezone, you may be pinged outside of the expected range.

Copy link
Member

@dhruvkb dhruvkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, works well!

In the media store, we set fetchingError to 404 status in case there are no results, to show the 404 page.

This seems wrong to me. If we have no results for a query, that does not imply 404 because the /search page itself still exists.

frontend/src/pages/search.vue Show resolved Hide resolved
@obulat
Copy link
Contributor Author

obulat commented Jul 25, 2023

This seems wrong to me. If we have no results for a query, that does not imply 404 because the /search page itself still exists.

Right, I haven't thought about it. Thank you, Dhruv!

@obulat obulat marked this pull request as draft July 25, 2023 05:36
@obulat obulat requested a review from dhruvkb July 25, 2023 06:34
@obulat obulat force-pushed the fix/media_error_handling branch 5 times, most recently from 10fb0c7 to 421bafc Compare July 26, 2023 13:33
@obulat obulat marked this pull request as ready for review July 26, 2023 14:14
@obulat
Copy link
Contributor Author

obulat commented Jul 26, 2023

LGTM, works well!

In the media store, we set fetchingError to 404 status in case there are no results, to show the 404 page.

This seems wrong to me. If we have no results for a query, that does not imply 404 because the /search page itself still exists.

I updated the PR to set the error to {message: NO_RESULTS} when we have no results so that we show the client-side No results page instead of the Nuxt yellow error page.

@obulat obulat force-pushed the fix/media_error_handling branch 2 times, most recently from 89efb8e to 989d8f5 Compare July 27, 2023 14:49
Copy link
Member

@dhruvkb dhruvkb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, thanks! I appreciate your quick turnaround on the requested changes.

@obulat obulat merged commit a102809 into main Jul 28, 2023
49 checks passed
@obulat obulat deleted the fix/media_error_handling branch July 28, 2023 06:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
💻 aspect: code Concerns the software code in the repository ✨ goal: improvement Improvement to an existing user-facing feature 🟧 priority: high Stalls work on the project or its dependents 🧱 stack: frontend Related to the Nuxt frontend
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

Improve error handling in the media store
4 participants