k4run: allow addition of additional parameters from steering files

[Zehvogel]

With this change you can add a parameter from a steering file by simply doing something like

parser.add_argument("-f", "--foo", type=int, help="hello world")
my_opts = parser.parse_known_args()
print(my_opts[0].foo)

this is achieved by making the parser global so that it is also passed to the exec() call. This might not be the most beautiful solution, but imo its the simplest and the use of implicitly appearing global variables in the current code is already a thing anyway...

BEGINRELEASENOTES

• k4run: allow addition of additional parameters from steering files

ENDRELEASENOTES

[jmcarcell]

But to get access to whatever value would be passed you need to have access to at least one more variable that you would have to make global, otherwise how do you access the value of foo? Note that we already can pass values corresponding to the parameters of algorithms or services, check

k4run steering_file.py --help

for example

$ k4run checkExampleEventData.py --help
usage: k4run [-h] [--dry-run] [-v] [-n NUM_EVENTS] [-l] [--gdb] [--ncpus NCPUS]
             [--EventDataSvc.OutputLevel [EVENTDATASVC.OUTPUTLEVEL]]
             [--EventDataSvc.input [EVENTDATASVC.INPUT]]
             [--EventDataSvc.FirstEventEntry [EVENTDATASVC.FIRSTEVENTENTRY]]
             [--PodioInput.OutputLevel [PODIOINPUT.OUTPUTLEVEL]]
             [--PodioInput.collections PODIOINPUT.COLLECTIONS [PODIOINPUT.COLLECTIONS ...]]
             [--k4FWCoreTest_CheckExampleEventData.OutputLevel [K4FWCORETEST_CHECKEXAMPLEEVENTDATA.OUTPUTLEVEL]]
             [--k4FWCoreTest_CheckExampleEventData.magicNumberOffset [K4FWCORETEST_CHECKEXAMPLEEVENTDATA.MAGICNUMBEROFFSET]]
             [config_files ...]

Run job in the Key4HEP framework

positional arguments:
  config_files          Gaudi config (python) files describing the job

options:
  -h, --help            show this help message and exit
  --dry-run             Do not actually run the job, just parse the config files
  -v, --verbose         Run job with verbose output
  -n NUM_EVENTS, --num-events NUM_EVENTS
                        Number of events to run
  -l, --list            Print all the configurable components available in the framework and
                        exit
  --gdb                 Attach gdb debugger
  --ncpus NCPUS         Start Gaudi in parallel mode using NCPUS processes. 0 => serial mode
                        (default), -1 => use all CPUs
  --EventDataSvc.OutputLevel [EVENTDATASVC.OUTPUTLEVEL], --OutputLevel.EventDataSvc [EVENTDATASVC.OUTPUTLEVEL]
                        output level [Service]
  --EventDataSvc.input [EVENTDATASVC.INPUT], --input.EventDataSvc [EVENTDATASVC.INPUT]
                        Name of the file to read [unknown owner type]
  --EventDataSvc.FirstEventEntry [EVENTDATASVC.FIRSTEVENTENTRY], --FirstEventEntry.EventDataSvc [EVENTDATASVC.FIRSTEVENTENTRY]
                        First event to read [unknown owner type]
  --PodioInput.OutputLevel [PODIOINPUT.OUTPUTLEVEL], --OutputLevel.PodioInput [PODIOINPUT.OUTPUTLEVEL]
                        output level [Gaudi::Algorithm]
  --PodioInput.collections PODIOINPUT.COLLECTIONS [PODIOINPUT.COLLECTIONS ...], --collections.PodioInput PODIOINPUT.COLLECTIONS [PODIOINPUT.COLLECTIONS ...]
                        Places of collections to read [PodioInput]
  --k4FWCoreTest_CheckExampleEventData.OutputLevel [K4FWCORETEST_CHECKEXAMPLEEVENTDATA.OUTPUTLEVEL], --OutputLevel.k4FWCoreTest_CheckExampleEventData [K4FWCORETEST_CHECKEXAMPLEEVENTDATA.OUTPUTLEVEL]
                        output level [Gaudi::Algorithm]
  --k4FWCoreTest_CheckExampleEventData.magicNumberOffset [K4FWCORETEST_CHECKEXAMPLEEVENTDATA.MAGICNUMBEROFFSET], --magicNumberOffset.k4FWCoreTest_CheckExampleEventData [K4FWCORETEST_CHECKEXAMPLEEVENTDATA.MAGICNUMBEROFFSET]
                        Integer to add to the dummy values written to the edm
                        [k4FWCoreTest_CheckExampleEventData]

[Zehvogel]

True oops, I did not try to use the value yet, I will rework this.

Regarding the other thing: I know about the possibility to pass parameters to the algorithms, but I want to have parameters to set which parts of a steering file should run

[Zehvogel]

This was quite tricky but the solution was simple.

With these changes it is now possible to add and parse arguments inside of steering files while keeping the help intact. Moreover, it also fixes the bug of

k4run --help file.py

not showing the Gaudi properties while

k4run file.py --help

did.

[jmcarcell]

Can you document that it is possible to do it and how to do it?

[Zehvogel]

Sure I can document it, can you point me to any k4run or steering file documentation where I should add it to?

[jmcarcell]

I'm not sure, the README maybe, adding a line or two about using k4run to run since we don't even have that (it's in the key4hep documentation but it's something very basic so I don't think it hurts having it here). And then adding parameters in the steering file, or everything in the key4hep documentation but then it's different for releases / nightlies so it's a bit annoying. I think for very specific details like adding a arguments inside the steering file the README is OK.

[Zehvogel]

I added some documentation to the README

[Zehvogel]

Superseded by: #134