Can't enable authentication in Docker image ("Please configure your webapp")

[nachogoro]

Description
I have made minor modifications to Dockerfile and docker/start.py from release 1.6.9 in order to enable authentication/authorization as described here.
Docker images in which start.py have been modified to use insert.xml as the insert file when deploying the .war file will fail to display OpenGrok in the browser:

There was an error!
The source root path has not been configured! Please configure your webapp.
Unable to determine source root path. Missing configuration?

Looking at the debug logs and comparing them to the unmodified image, it looks like we keep getting status 500 in wait_for_tomcat(), and we never start the sync.

To Reproduce

• Create a directory on your machine to act as the source directory for Opengrok projects and add a foo project:
  $ mkdir -p ~/projects-opengrok/foo; echo "Hello, world" > ~/projects-opengrok/foo/README

• Go to the opengrok repository and checkout release tag 1.6.9: $ git checkout -b 1.6.9

• Apply the following (fairly minor) patch: 0001-Enable_auth.txt

• Build the docker image: $ docker build -t opengrok-auth .

• Run a container on that image, with the right volume mapping:
  $ docker run -d -v ~/projects-opengrok/:/opengrok/src -p 8080:8080 -e OPENGROK_LOG_LEVEL=DEBUG opengrok-auth

• Open the browser, go to localhost:8080 and see the error message about the missing webapp configuration.

Additional information
Here are some debug logs from docker logs: logs.txt

[vladak]

See why the webapp returns the error - try bumping the Tomcat/OpenGrok log level in /usr/local/tomcat/conf/logging.properties. My initial guess was that the request done in wait_for_tomcat() fails because of the authentication check, however the 500 error and the actual message displayed means this has to be caused by something else. Also, it might be useful to check the web.xml and read-only config in the Docker container itself.

Another piece of advice would be to set this up standalone first, i.e. not in Docker.

[vladak]

Looking at the Dockerfile, these lines stick out:

ENV READONLY_CONFIG_FILE /opengrok/etc/readonly-configuration.xml
ENV INDEXER_OPT "-R $READONLY_CONFIG_FILE"

What the INDEXER_OPT does is that it will pass the bare read-only configuration to per project indexer, possibly overriding the code in

opengrok/tools/src/main/python/opengrok_tools/reindex_project.py

Line 123 in 52ef487

indexer_options = ['-R', conf_file] + args.options

which is meant to get fresh configuration from the web app so that the indexer has information about projects and repositories.

It should be sufficient to use just the READONLY_CONFIG_FILE so that the code in

opengrok/docker/start.py

Lines 521 to 543 in 52ef487

	#
	# If there is read-only configuration file, merge it with current
	# configuration.
	#
	read_only_config_file = os.environ.get('READONLY_CONFIG_FILE')
	if read_only_config_file and os.path.exists(read_only_config_file):
	logger.info('Merging read-only configuration from \'{}\' with current '
	'configuration in \'{}\''.format(read_only_config_file,
	OPENGROK_CONFIG_FILE))
	out_file = None
	with tempfile.NamedTemporaryFile(mode='w+', delete=False,
	prefix='merged_config') as tmp_out:
	out_file = tmp_out.name
	merge_config_files(read_only_config_file, OPENGROK_CONFIG_FILE,
	tmp_out, jar=OPENGROK_JAR, loglevel=log_level)
	if out_file and os.path.getsize(out_file) > 0:
	shutil.move(tmp_out.name, OPENGROK_CONFIG_FILE)
	else:
	logger.warning('Failed to merge read-only configuration, '
	'leaving the original in place')
	if out_file:
	os.remove(out_file)

does the right thing.

[nachogoro]

Thanks for looking into this. Unfortunately, removing the INDEXER_OPT line doesn't seem to change much.
These are the logs after removing the INDEXER_OPT environment variable and increasing the logging level to FINEST, which don't seem to provide that much information (at least to me!)
debug-logs.txt

[vladak]

Yep, the INDEXER_OPT would affect the indexer only.

[nachogoro]

web.xml (from /usr/local/tomcat/webapps/ROOT/WEB-INF/web.xml) is here: web.xml.txt

I can see that the added insert.xml bit does not have the XML namespace, but it doesn't seem to make a difference.

web.xml (from /usr/local/tomcat/conf/web.xml) is here: conf-web.xml.txt
It does not include the insert.xml bit. I'm not sure if it should: on the one hand it is not mentioned in the docs as the .xml to be modified (and it isn't modified by the deploy_war function), but on the other, I tried adding the insert.xml stuff here manually on a previous build a few days ago by accident and it seemed to fix the issue (but I had been tinkering way too much to know whether it had any effect or not).

readonly-configuration.xml is here, seems to be intact: readonly-configuration.xml.txt

[vladak]

What is inside /opengrok/etc/configuration.xml ? Normally when the webapp is deployed, the following log messages will be displayed:

opengrok-latest | 19-May-2022 07:20:00.378 INFO [main] org.opengrok.web.WebappListener.contextInitialized Starting webapp with version 1.7.31 (6333fd74002ce5c6b4229b0c7460bd5acab9adec)
opengrok-latest | 19-May-2022 07:20:00.378 INFO [main] org.opengrok.indexer.configuration.Configuration.read Reading configuration from /opengrok/etc/configuration.xml
opengrok-latest | 19-May-2022 07:20:05.499 INFO [main] org.opengrok.indexer.util.Statistics.logIt Done invalidating 14 repositories (took 5.58 seconds)
opengrok-latest | 19-May-2022 07:20:05.509 WARNING [main] org.opengrok.indexer.framework.PluginFramework.reload Plugin directory not found or not readable: /opengrok/data/../plugins. All requests allowed.
opengrok-latest | 19-May-2022 07:20:05.952 INFO [Thread-18] org.opengrok.suggest.Suggester.init Initializing suggester
...

I do not see this in the debug-logs.txt.

Again, I'd suggest to set this up in standalone environment first and then maybe go step by step.

[nachogoro]

These are the contents of configuration.xml: configuration.xml

Really appreciate the help and the suggestions. Unfortunately, I don't think I'll have the time to try to set it up standalone for a few days (hence why I was hoping I could quickly configure the Docker image with authentication for quick deployment).

[vladak]

The configuration seems to be valid - it checks out syntactically and the contents look sound (for a bare config, that is).