enhance batch output for cluster runs

[blaskovicz]

For non-interactive (eg: batch) environments where puppeteer-cluster is used, it would be nice to enhance the display of statistics to be more friendly:

Example output:

[K== Start:     2020-03-26 20:01:45.859
�[K== Now:       2020-03-26 20:01:50.859 (running for 5.0 seconds)
�[K== Progress:  0 / 306 (0.00%), errors: 0 (0.00%)
�[K== Remaining: unknown (@ 0 pages/second)
�[K== Sys. load: 7.3% CPU / 94.3% memory
�[K== Workers:   4
�[K   #0 WORK UNKNOWN TARGET
�[K   #1 WORK UNKNOWN TARGET
�[K   #2 WORK UNKNOWN TARGET
�[K   #3 WORK UNKNOWN TARGET

Ideally, there would be 3 enhancements made (listed in order of request priority):

1. ability to aggregate display output (for example, when tasks change state or on some interval)
2. ability to set custom work target descriptions (eg: for when the data passed to a task is a non-string (code here);
3. ability to detect non-interactive terminal and avoid outputting terminal escape sequences

[thomasdondorf]

I guess it's better to just implement a API to read the statistics (see #8)? Then people can develop their own code to log the status to console.

I'll leave it open for discussion.

[BrightSoul]

About item 2 (work target descriptions): the worker can update the url property of the complex object you passed as data, and it will display as the worker description. It doesn't need to be an actual url at all. It just needs to be a string, as you can see here in the source.

puppeteer-cluster/src/Job.ts

Lines 31 to 42 in fbd9a9f

	public getUrl(): string | undefined {
	if (!this.data) {
	return undefined;
	}
	if (typeof this.data === 'string') {
	return this.data;
	}
	if (typeof (this.data as any).url === 'string') {
	return (this.data as any).url;
	}
	return undefined;
	}

It's kind of hacky but... well, it works for me. Displaying descriptions is essential in my case because each of my workers performs a long-running, multiple step task. And here's the output.

You can even use ANSI color escape codes.

data.url = '\033[33m Doing step 1 \033[39m';