Performance: Server-side-like pagination via web worker (step 4)

[nwmac]

Instead of holding all resources in the VueX store, only hold the resources for the current page in a list.

A web worker presents a paginated resource UI to the UI - it fetches all resources of a given type and processes the web socket messages to keep those up to date.

When a list is requested by the UI, the web worker will sort, filter and paginate the resources required and return this back to the UI - which can store if in the VueX store.

When updates come in, the web worker processes those, determines if they affect any active pages, and if so, sends the updates to the UI which then updates the VueX store, resulting in the same reactivity we have now, so the list will auto update.

When a user leaves a page, the lists on it are removed and the web worker stops sending updates through - but keeps processing updates for the resources it holds (We may choose not to keep all resources in memory at a later time).

Additional context

• Step 1 is Performance: Move cluster based socket work into web worker (step 1) #7894
• Step 2 is Performance: Only update store if a resource type has changed (step 2) #7895
• Step 3 is Performance: Make API requests from within the web worker (step 3) #7896
• This is step 4

[catherineluse]

@Sean-McQ I had a question about the web worker - does it store "dehydrated" or "rehydrated" data? I was looking into the possibility of doing less hydration - maybe trying to only use this hydration code on a component by component basis (https://github.com/rancher/dashboard/blob/master/shell/plugins/dashboard-store/index.js#L98). And I couldn't remember if the web worker project is already doing that

[ghost]

@catherineluse I'm not hydrating the list resources inside of the worker but that is baked into the mutation I use to load it into the store when I pass it to the UI thread. We could potentially bake the hydration step into the getter so that we're not storing the full class instances in there or preferably only instantiate the various row resources in the component that requests it and avoid having the store hydrate anything (in the case of lists).

[gaktive]

This ticket will be split to allow for the work for the Q1 release to be deliverable but then have additional work for a subsequent release.

[richard-cox]

Edit - Snip. See #8527

[richard-cox]

Given change in direction, bumping out to 2.7 Q3. We should look at this work alongside #8507

[gaktive]

Need to re-evaluate this ticket as part of our existing performance work. This didn't land but the backend plans changed.

[nwmac]

May not do this if we can get server-side pagination

[richard-cox]

No longer applicable