Add interactive -i flag to k4run.

[scott-snyder]

With -i, k4run will start a Python command loop after reading the configuration file, allowing one to inspect the configuration interactively. This is sometimes useful for debugging.

BEGINRELEASENOTES

• k4run: add the switch --interactive, -i. When given, k4run will enter a Python command loop after the configuration has been read, allowing one to inspect the configuration interactively. This is useful for debugging.

ENDRELEASENOTES

[tmadlener]

This looks very useful indeed. I think it would need some documentation on how to kick off Gaudi after inspection.

[jmcarcell]

I don't understand very well the purpose of this. All the options of the algorithms will be printed before running, and there is --dry-run to print them and exit, then you can find whatever you want from that output. In the current version of the PR, the dictionaries containing the options are not available so it doesn't look trivial to me how to do anything with the variables.
Also running

python -m pdb $(which k4run) steering.py
...
b 286
c
p opts_dict

does what I think you want to do? There is interact to jump to a python interpreter, meaning that we could simply set optionally a breakpoint.

[scott-snyder]

This is modeled after similar functionality available at ATLAS. It's
not something one uses every day, but having it available has certainly
saved me time on numerous occasions when trying to understand what's
going on with a job configuration.

As to --dry-run: the value of dumps decreases as the complexity of the
configuration increases. k4 configurations i've seen have maybe 10 or 20
components --- but an ATLAS job may configure a couple thousand components.
It's often easier then to interactively pick dump what one is interested in
rather than trying to pick it out of a huge dump. Also, job configurations
typically contain conditionals, functions, etc., the state of which
is not captured at all in a static dump.

pdb could be used here, i guess, but it's not as convenient.
It's also not compatible with the execute-time interactive mode provided
in ATLAS, where the event loop can be driven from python and the contents
of the event store inspected interactively. That's not close to working
in k4, but might be something to look towards.

In practice, i added this in the process of debugging a tricky Gaudi
configuration problem that was affecting the allegro calorimeter
configuration
https://gitlab.cern.ch/gaudi/Gaudi/-/merge_requests/1759
for which it was useful to interactively poke at the Gaudi configuration
internals. Since as i said i've occasionally found the analogous
ATLAS feature quite useful, i thought it would be nice to try
to make it available for others.

[tmadlener]

I like the interactive flag over the pdb option also because people don't necessarily know about pdb, but interactive is pretty self-explanatory. Additionally, with pdb you have to know where to break, which usually boils down to figuring out the line number before hand or stepping over a bunch of imports until the function name that you want to break on becomes known which, again, involves knowing at least some of the internals, which you can completely skip with interactive as that drops you into an interpreter at a reasonable spot for most cases.

What might be useful is another small utility function that kicks off the main gaudi loop after people are done inspecting and potentially fixing things up immediately, I think. But that can also be done later, if and once more people want to have it.

[scott-snyder]

Resolved conflict and squashed.

[tmadlener]

For me this is good to be merged.

[jmcarcell]

Can you add in doc/k4run-args how this is supposed to be used (i.e. print(alg) gives you the details about the options).

What might be useful is another small utility function that kicks off the main gaudi loop after people are done inspecting and potentially fixing things up immediately, I think. But that can also be done later, if and once more people want to have it.

This is what happens when one leaves the interpreter.