HTML response may be too large/long to generate

[pipoprods]

When a page generates a LDAP query that returns a huge number of records, then all of them are processed on the backend to generate a HTML page containing all the results. Pagination is completely handled on the frontend side, hiding/showing elements corresponding to the page number.

There are drawbacks to this approach:

• when the HTML page is too long to generate, the request may reach reverse-proxies timeout
• the response may be very huge (measured >50MB)

The pagination system architecture should be reconsidered to only return current page of results and request the backend at each page change.

[coudot]

See also ltb-project/white-pages#1

[davidcoutadeur]

Some thoughts:

• naïve approach: each time a search is requested, read every entry in the LDAP directory. This request can be slow (~5s for 5000 entries returned)
• store the results in the user session (php session). advantage: when another user add or modify an entry while a first one is doing a paginated search, the one doing the search keeps having a coherent view of the results
• store the result in a global cache (shared between users). Drawbacks: deal with the synchronization issues between cache and directory

[coudot]

Another related issue: #207