Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: improve perf inside search #2663

Closed
wants to merge 9 commits into from
+336 −92
Closed

perf: improve perf inside search #2663

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
3a27c1b
feat(deps): Upgrade cozy-bar to be able to call onSelect function
trollepierre Aug 10, 2022
f1e5acf
feat(deps): Upgrade cozy-client to be able to findAll in FileCollection
trollepierre Aug 10, 2022
1133fb9
feat(search): improve search by querying fewer data
trollepierre Aug 4, 2022
8d4abc3
chore(bundlemon): Upgrade maxSize of qualification migration
trollepierre Aug 12, 2022
298411d
fix(search): use find method where args are for find
trollepierre Aug 17, 2022
ba47d2e
fix(search): note needs metadata to be considered as note
trollepierre Aug 18, 2022
4dcbc83
fix(search): notes function can not be postMessage as a function
trollepierre Aug 18, 2022
db97874
fix(search): isNote should not check all metadata, missing while fetc…
trollepierre Aug 22, 2022
1cd1efc
fixup: with cozy-bar upgrade
trollepierre Aug 23, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 10 additions & 1 deletion src/drive/web/modules/queries.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -343,7 +343,16 @@ export const prepareSuggestionQuery = () => ({
$or: [{ $exists: false }, { $eq: false }]
}
},
fields: ['_id', 'trashed', 'dir_id', 'name', 'path', 'type', 'mime'],
fields: [
'_id',
'trashed',
'dir_id',
'name',
'path',
'type',
'mime',
'metadata'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Une chose est sûre, c'est qu'on peut pas garder metadata au complet. Soit on arrive à aller lire quelques champs particuliers, soit on change la façon de checker si c'est une note. Car metadata sur les notes, ça peut vite peser des centaines de ko par note.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

j'ai recréé un helper isNote, car les metadata peuvent être volumineuses.

Copy link
Contributor

@Crash-- Crash-- Aug 22, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know if I like this idea.

You can add to the select(['metadata.title', 'metadata.version']) . title + version are not so big. So the helper can be if(metata.title && metadata.version && file.endsWith... && ...) .

See a test query with metadata.datetime to see the result.

Capture d’écran 2022-08-22 à 11 31 33

What do you think?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have only one note in my local Drive.
When adding:

    'metadata.content',
    'metadata.schema',
    'metadata.title',
    'metadata.version'

as fields, the size of request increase
from 650ko / 39,87 ko transferred in 538ms
to 657,97 ko / 41,61ko transferred in 578ms.

Capture d’écran 2022-08-22 à 12 03 37
Capture d’écran 2022-08-22 à 12 05 08

That is quite huge, I think.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not what I suggested. I suggested to only add version + title. Not schema + content.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perfect
Capture d’écran 2022-08-22 à 14 11 58
PR updated

],
indexedFields: ['_id'],
limit: 1000
})
Expand Down
11 changes: 10 additions & 1 deletion src/drive/web/modules/services/components/SuggestionProvider.spec.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,16 @@ describe('SuggestionProvider', () => {
// Then
expect(mockClient.collection).toHaveBeenCalledWith('io.cozy.files')
expect(mockFindAll).toHaveBeenCalledWith(null, {
fields: ['_id', 'trashed', 'dir_id', 'name', 'path', 'type', 'mime'],
fields: [
'_id',
'trashed',
'dir_id',
'name',
'path',
'type',
'mime',
'metadata'
],
indexedFields: ['_id'],
limit: 1000,
partialFilter: {
Expand Down