Issue/1508 pagination #1541

cayb0rg · 2023-10-31T19:05:28Z

Reopens #1518
Fixes #1508

Adds pagination to the admin interface's instance search and admin user search
Adds offset, limit, and addtional order_by ID to SQL query for instances in My Widgets Page
- This is also applied to the collaborator and extra attempts dialog
Adds scrollbar to collaborator dialog user search results
Set avatar sizes in user admin search to 50x50

~~Currently, the user and instance search retrieve all entries in the database on page load. Depending on feedback, this may switch to fetching only after someone has typed something.~~ Fixed.

… image size of user avatars in search results

clpetersonucf

Good work so far 👍, some initial feedback items as a first pass

clpetersonucf · 2024-01-23T21:30:19Z

fuel/app/classes/materia/widget/instance/manager.php

@@ -14,7 +14,7 @@ static public function get($inst_id, $load_qset=false, $timestamp=false, $delete
 		return count($instances) > 0 ? $instances[0] : false;
 	}

-	static public function get_all(Array $inst_ids, $load_qset=false, $timestamp=false, bool $deleted=false): array
+	static public function get_all(Array $inst_ids, $load_qset=false, $timestamp=false, bool $deleted=false, $offset=0, $limit=2147483647): array


Nice Y2K38 protection 💪

clpetersonucf · 2024-01-23T21:55:20Z

src/util/api.js

 */
-export const apiSearchWidgets = input => {
+export const apiSearchWidgets = (input, page_number) => {


A bit of a nitpick, but I think it's important to distinguish widgets from instances when it comes to API-related methods. Since this is searching for instances, it should be named as such.

clpetersonucf · 2024-01-23T21:56:50Z

fuel/app/classes/controller/api/admin.php

@@ -69,13 +69,20 @@ public function post_user($user_id)
 		return \Service_User::update_user($user_id, $user);
 	}

-	public function get_widget_search(string $input)
+	public function get_widget_paginated_search(string $input, string $page_number)


I made this comment in the front-end's api file too, but if we're looking for instances and not widgets, the method name should explicitly use the term instance.

clpetersonucf · 2024-01-23T22:09:36Z

src/components/hooks/useInstanceList.jsx

 import { iconUrl } from '../../util/icon-url'

-export default function useInstanceList() {
+export default function useInstanceList(userOnly = true, query = "") {


The more I think about this, the more I want this hook to be split into two different instance list hooks, because while there are some similarities, there are some key differences:

They're hitting different API endpoints

They're both requesting lists of widgets, but for different purposes. This especially affects the widgets query cache: one is a list of instances the user owns, the other is a list of instances the user may not expressly have access to. IMO, the widgets query cache should remain the list of widgets we own (in the My Widgets page) and the new hook would use a different query key.

It makes references to the hooks elsewhere much cleaner as to what exactly the hook is being used for

clpetersonucf · 2024-01-24T14:48:04Z

Additional feedback: loading the entire list of users and instances on admin page load is a definite no-go. That's a very expensive request and there's no good reason for it. I'd much rather exclusively return the results of search queries instead.

cayb0rg · 2024-01-25T18:00:15Z

New changes: updated the function names to use "instances" instead of "widgets", created useSearchInstances hook from useInstanceList, and prevents searching on page load in admin search panel.

clpetersonucf

The only thing I'm seeing is the user panel continues to fetch the entire table's contents on load, which I'm not a fan of. It looks like the infinite query is missing the enabled: query.length > 0 param present in the instance search query.

The limit in the instance manager's get_all method is fine, generally, but I'm not sure when it would realistically (or should realistically) be used. Prior to the reach branch, superusers would have implicit ownership of every instance the database, and thus the API would try and respond with the entire table when visiting My Widgets (because the request wasn't paginated). That request actually caused PHP-FPM to run out of memory. I think I'm fine leaving it in, but it's very much a "you probably shouldn't ever request every instance in the database in one go" situation.

Other than that: LGTM 💪

cayb0rg · 2024-01-31T15:25:44Z

That's a good point; functions querying the widget_instance table are either fetching a page or only the first instance in the results, so it's safe to leave the default limit at 80.

Caught an issue with paginated results returning duplicate entries because of the ORDER BY keyword; removing this seems to have fixed it.

clpetersonucf

Three small requests to change the manner in which you're calculating $has_next_page; other than that it's good!

clpetersonucf · 2024-01-31T21:41:34Z

fuel/app/classes/materia/widget/instance/manager.php

+		$displayable_items = self::get_all($inst_ids, false, false, false, $offset, $items_per_page);
+
+		// if the returned number of instances is greater than a page, there's more pages
+		$has_next_page = sizeof($displayable_items) > ($items_per_page - 1) ? true : false;


This is getting into the weeds a bit, but this method of determining whether there's another page will still return true if the total item count is divisible by items_per_page.

A better option might be to set the query limit to $limit + 1, and simply check whether $displayable_items is greater than $items_per_page. That shouldn't affect the offset, since the +1 isn't part of the calculation.

Update: I'm also wondering if the behavior you noticed with order by was influenced by the +1 offset. I'd prefer to keep the order by clause when performing the query, so it'd be worth double-checking after updating these.

Having a single order_by seems to have been causing the issue, since multiple instances can have the same created_at value. I added an additional criteria to break up ties by ordering by ID, and reinstated the original pagination logic (adding +1 to the limit and then popping the last entry).

clpetersonucf · 2024-01-31T21:42:09Z

fuel/app/classes/materia/widget/instance/manager.php

+		$displayable_items = self::get_widget_instance_search($input, $offset, $items_per_page);
+
+		// if the returned number of instances is greater than a page, there's more pages
+		$has_next_page = sizeof($displayable_items) > ($items_per_page - 1) ? true : false;


Same thought process as my comment on instance/manager line 91.

clpetersonucf · 2024-01-31T21:44:08Z

fuel/app/classes/materia/api/v1.php

+		// query DB for only a single page + 1 item
+		$displayable_items = \Model_User::find_by_name_search($input, $offset, $items_per_page);
+
+		$has_next_page = sizeof($displayable_items) > ($items_per_page - 1) ? true : false;


Same feedback here as my comment in instance/manager line 91.

clpetersonucf

The second order by clause addition is a nice touch. Nicely done.

This looks good, I think it's ready to ship.

cayb0rg and others added 11 commits October 11, 2023 12:21

piggyback on my-widgets-page pagination hook

e35432c

Merge master into issue/1508-pagination

e94da59

Merge branch 'dev/10.0.1' into issue/1508-pagination

7b87acb

Increase pagination speeds and add loading icon

9c128bb

Merge branch 'dev/10.0.1' into issue/1508-pagination

1fa95ab

Apply pagination rules to all components using apiSearchUsers and fix…

7c1e441

… image size of user avatars in search results

Update tests for search user

afeebca

Remove log

6d754c9

User search fix and collaborator results scroll

5a81bff

Remove user tests

ac422c9

Merge in dev/10.1.0

bf51d03

clpetersonucf requested changes Jan 23, 2024

View reviewed changes

cayb0rg and others added 2 commits January 25, 2024 12:52

split hooks, rename funcs + prevent search on page load

e3427cf

Merge branch 'dev/10.1.0' into issue/1508-pagination

486d3b4

clpetersonucf requested changes Jan 30, 2024

View reviewed changes

cayb0rg added 3 commits January 31, 2024 08:17

Merge branch 'dev/10.1.0' into issue/1508-pagination

6e3ab86

Fix issue with double instances and reduce default limit

d735b60

Reduce another default limit

6e1bf19

clpetersonucf requested changes Jan 31, 2024

View reviewed changes

cayb0rg added 3 commits February 1, 2024 09:06

Add additional order_by query to get predictable results

0c9200e

Remove unnecessary front-end sort

c3549e0

Add test

7598e84

clpetersonucf approved these changes Feb 1, 2024

View reviewed changes

clpetersonucf merged commit ff99e23 into ucfopen:dev/10.1.0 Feb 1, 2024
2 checks passed

clpetersonucf mentioned this pull request Feb 1, 2024

Dev/10.1.0 #1545

Merged

clpetersonucf mentioned this pull request Mar 27, 2024

User and Instance admin search results should be paginated #1508

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue/1508 pagination #1541

Issue/1508 pagination #1541

cayb0rg commented Oct 31, 2023 •

edited

Loading

clpetersonucf left a comment

clpetersonucf Jan 23, 2024

clpetersonucf Jan 23, 2024

clpetersonucf Jan 23, 2024

clpetersonucf Jan 23, 2024

clpetersonucf commented Jan 24, 2024

cayb0rg commented Jan 25, 2024

clpetersonucf left a comment

cayb0rg commented Jan 31, 2024

clpetersonucf left a comment

clpetersonucf Jan 31, 2024

clpetersonucf Jan 31, 2024

cayb0rg Feb 1, 2024

clpetersonucf Jan 31, 2024

clpetersonucf Jan 31, 2024

clpetersonucf left a comment

Issue/1508 pagination #1541

Issue/1508 pagination #1541

Conversation

cayb0rg commented Oct 31, 2023 • edited Loading

clpetersonucf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clpetersonucf commented Jan 24, 2024

cayb0rg commented Jan 25, 2024

clpetersonucf left a comment

Choose a reason for hiding this comment

cayb0rg commented Jan 31, 2024

clpetersonucf left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

clpetersonucf left a comment

Choose a reason for hiding this comment

cayb0rg commented Oct 31, 2023 •

edited

Loading