Optimize workspace tree building

[bloodearnest]

Previously, we walked the file tree as we recursed down, calling
iterdir(), is_dir(), and is_file() on the way. For large workspace
trees, this was very slow.

Instead, we now get a list of all relative paths at the start,
and then walk down the paths, just processing them as strings, not
hitting disk. This is similar to how we built the tree for each
filegroup before, from a flat list of paths in the db. So that method
had been extracted as get_path_tree(), made more general, and is now
used by both get_workspace_files and get_request_files.

As we do not want to call is_dir() to detect directories, we instead
use a heurstic based on the path.

• if it has descendant paths, it is a directory
• if it has no descendants and no suffix, it is a directory.

This heuristic is not perfect - an empty directory with a . in the
name will be misclassified as a file. However, this case seems very
rare, and I have added some protection in this case so we fail
gracefully.

[rebkwok]

LGTM. The UI doesn't quite work as usual if there is an empty directory with an extension - it gets displayed with the folder icon but no arrow (unlike a normal empty directory). But I don't think it's worth spending any time fixing.

[bloodearnest]

LGTM. The UI doesn't quite work as usual if there is an empty directory with an extension - it gets displayed with the folder icon but no arrow (unlike a normal empty directory). But I don't think it's worth spending any time fixing.

Huh, for me it displayed as a file, ie not directory icon, and no triangle to open. When I click on it, I see the text I expect

I think this case is very unlikely, so am merging as is.