[upload] Add support for uploading folders

[onuridrisoglu]

It should be possible to select or drop a folder and upload the contents of the folder

[Haprog]

This is a good idea to offer a slightly better UX for uploading all files in a directory. Can be considered as progressive enhancement imo and ok to implement even if it won't work on all browsers (looks like it's well supported on desktop browsers except not on IE11).

Related resources:

• https://caniuse.com/#feat=input-file-directory
• https://wicg.github.io/directory-upload/proposal.html
• https://wicg.github.io/entries-api/#dom-htmlinputelement-webkitdirectory

[Haprog]

One thing to consider is what should happen if the selected directory contains nested directories with more files (and possibly files of same name in sub directories)? I didn't check if the native APIs allow for getting the whole directory hierarchy or only some flattened list of files from the directory (or does it even get files from sub directories of selected directory?).

There could be an additional (optional) related feature that would automatically zip the directory on client side before uploading it. Or other special handling (or customizable handler method) might be needed.

We should probably investigate how it behaves in these cases with the native:

<input type="file" directory>

and try to mimic that behaviour. Possibly providing additional value/features.

[web-padawan]

Note, we can use webkitdirectory attribute for uploading folders - see the Codepen illustrating how it works.
While it allows to select folders, it disallows selecting individual files, so you can't have both at the same time.

[web-padawan]

Here's the current version of the File and Directory Entries API: https://wicg.github.io/entries-api/

It contains some examples of how to get the directories from the drag and drop (which will require us to change how we handle event.dataTransfer object) and helper functions e.g. getEntriesAsPromise() and readFileEntry().

These APIs could be used to get the contents (individual files) from the folder uploaded using drag & drop.

[tommilukkarinen]

Had some fun learning JS and injecting code into Vaadin client:

Problems: Does not pass folder structure to server, might be easiest to make with standard element.$server invocation

Test with text files, these will print to the screen

Tried only in Jetty, might be some problems in production release?

Use:

1. press inject button
2. drop folders to upload area

package fi.protieto.juuri.front.pages.map4;

import com.vaadin.flow.component.button.Button;
import com.vaadin.flow.component.html.Div;
import com.vaadin.flow.component.upload.Upload;
import com.vaadin.flow.component.upload.receivers.MultiFileMemoryBuffer;
import com.vaadin.flow.router.Route;
import org.apache.commons.io.IOUtils;

import java.io.InputStream;
import java.nio.charset.StandardCharsets;

// written on top of Vaadin autoupload example

@Route("upload-auto-upload-disabled")
public class AutoUpload extends Div {
	// event listener explained
	// https://developer.mozilla.org/en-US/docs/Web/API/EventTarget/addEventListener
	// handling folders explained
	// https://stackoverflow.com/questions/3590058/does-html5-allow-drag-drop-upload-of-folders-or-a-folder-tree
	// replacing functions explained:
	// https://stackoverflow.com/questions/2136522/can-you-alter-a-javascript-function-after-declaring-it
	// web component:
	// https://github.com/vaadin/web-components/blob/main/packages/upload/src/vaadin-upload.js
	// source in chrome:
	// vaadin -> bundles -> node-modules -> @vaadin -> upload -> src -> vaadin-upload.js

	public AutoUpload() {
		MultiFileMemoryBuffer buffer = new MultiFileMemoryBuffer();
		Upload upload = new Upload(buffer);
		add(upload);

		Button injectJS = new Button("inject");
		injectJS.addClickListener(e -> {
			upload.getElement().executeJs(
							"" +

							"function traverseFileTree(item, path, uploadThing) {" +
								"if (item.isFile) {" +
									"item.file(function(file) {" +
										"uploadThing._addFile(file);" +
									"});" +
								"}" +
								"if (item.isDirectory) {" +
									"var dirReader = item.createReader();" +
									"dirReader.readEntries(function(entries) {" +
										"for (var i=0; i<entries.length; i++) {" +
											"traverseFileTree(entries[i], path + item.name + '/', uploadThing);" +
										"}" +
									"});" +
								"}" +
							"}" +

							"this.addEventListener('drop', function(event) {" +
								"var items = event.dataTransfer.items;" +
								"console.log('_onDrop top level item size: ' + items.length);" +
								"for (var i=0; i<items.length; i++) {" +
									"var item = items[i].webkitGetAsEntry();" +
									"var length = traverseFileTree(item, '', this);" +
								"}" +
							"});" +

							"this._addFiles = function(files) {" +
								"if(!files.isArray) {" +
									"console.log('_addFiles files as string (not array):' + JSON.stringify(files));" +
									"return;" +
								"}" +
							"};"
					);
		});

		add(injectJS);

		upload.addSucceededListener(e -> {
			try {
				InputStream is = buffer.getInputStream(e.getFileName());

				String result = IOUtils.toString(is, StandardCharsets.UTF_8);

				System.out.println(e.getFileName() + ": " + result);

				is.close();
			}catch (Exception ex){
				ex.printStackTrace();
			}
		});


	}
}

[rolfsmeds]

If it's not possible to support both files and folders in the native file selector, then we'll just have to accept that, and document that switching to folder mode means you can only upload files thru drag and drop, and that if you need to support both through the native selector, you'll have to provide a toggle for switching between file and folder upload.

[sissbruecker]

Support for dragging and dropping folders has been added with #8032.

For selecting a directory through the native file dialog there is a draft PR here: #8057.

As the implementation involves some opinionated choices and browser limitations, we want to outline the planned implementation here in order to possibly get some feedback:

• In order to allow uploading a folder through the native file dialog, the component needs to be configured into directory mode
• In that mode users can only select a single folder, but not files or multiple folders - that is a browser limitation that can not be changed
• After selecting a folder, all browsers currently show a confirmation dialog to users where they must confirm that they understand that all contents of that folder will be uploaded
• In that mode it would still be possible to drag and drop a combination of files and folders. We assume that selecting a folder is mostly a shortcut for selecting multiple files, as such it doesn't seem useful to prevent users from doing that.
• Selecting a folder will recursively add all files from that folder and nested folders to the upload list
• The folder structure would not be preserved, the name for each uploaded file will just contain the file name, but not the folder hierarchy
• Files are still filtered by the accept filter, the maxFiles file limit is still respected if configured

If you have any feedback regarding the proposed implementation please leave a comment.

[jorgheymans]

Interesting ! Did you consider some sort of upload queueing, so that uploading a directory with hundreds of large files does not swamp the server ?

[sissbruecker]

Nope, but that seems like a good idea. However I think this is a separate issue and that problem already exists today as you can select an unlimited number of files. So that will probably not make it into a first version of this feature.

[knoobie]

Mentioned issue: #6698

[rolfsmeds]

I presume we can close this now, and leave file-upload-dialog support as a separate issue?

[sissbruecker]

I was planning to keep this open until selecting folders through the dialog is resolved. We should be able to continue with this PR soon: #8057