progress bar

[jesteria]

No description provided.

[JordanHolland]

An option that i'm aware of is tqdm. Any other ideas?

[jesteria]

I think tqdm is probably the best bet.

I also see for example the library progress; but, tqdm looks more complete to me.

(If need be we could also certainly write our own.)

I think the biggest issue is that I'm not aware of libraries that control stdout in such a way that you can freely write to it without conflict with the progress bar. So, for example, tqdm recommends monkey-patching sys.stdout to itself. Not at all a big deal, but worth considering. (And, we might want to clean up what we output, more generally, first.)

Beyond that, we could get a simple integration out the door pretty easily, since the nprintML runner iterates over pipeline steps.

An improvement could be integrating the progress bar more deeply, such that steps can produce smaller increments as they progress – this would be most useful (and easiest) within PCAP ingestion – but, that's not so immediately necessary.

[JordanHolland]

tqdm is probably our best bet as it can slot in to a process pool without any code. I'm mostly convinced we just want a progress bar when we're actually running nPrint (i.e. 5/500 files being processed) and not the entire pipeline. We have no real access to the progress of AutoML beyond its own output.

[jesteria]

Sure – certainly for the time being, at least. (I'm not sure how we would reach into AutoGluon either.)

…

On Mon, Mar 15, 2021 at 8:07 AM Jordan Holland ***@***.***> wrote: tqdm is probably our best bet as it can slot in to a process pool without any code. I'm mostly convinced we just want a progress bar when we're actually running nPrint (i.e. 5/500 files being processed) and not the entire pipeline. We have no real access to the progress of AutoML beyond its own output. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#28 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEBUNWVIVQBTVKQ567RKXTTDYBAXANCNFSM4UVQRBGA> .