is it possible to use this with private pypi repositories?

[dogfish182]

I'm having trouble getting turbolayers to work when using a private pypi repo. Chosen dependency management is pipenv

the lock file contains

{
                "name": "private",
                "url": "${PIP_EXTRA_INDEX_URL}",
                "verify_ssl": true
            }

with PIP_EXTRA_INDEX_URL being exposed on the environment

attempting to do something like this

self.packager = PythonDependencyPackager(
            self,
            "Packager",
            runtime=Runtime.PYTHON_3_10,
            type=DependencyPackagerType.LAMBDA,
            preinstall_commands=[
                "export PIP_EXTRA_INDEX_URL='https://credential:[email protected]/artifactory/api/pypi/pypi-001/simple'"
            ],
        )

will generate the error

thing | 10:53:38 AM | CREATE_FAILED        | Custom::LayerPackager       | Packager/lambda_packager/Layer Packager/Default (PackagerlambdapackagerLayerPackagerB134287E) Received response status [FAILED] from custom resource. Message returned: COMMAND FAILED PIPENV_VENV_IN_PROJECT=1 pipenv sync
...[pipenv.exceptions.InstallError]: WARNING: Location '${PIP_EXTRA_INDEX_URL}/somepackage/' is ignored: it is either a non-existing path or lacks a specific scheme.
[pipenv.exceptions.InstallError]: ERROR: Could not find a version that satisfies the requirement somepackage==4.0.5 (from versions: none)
[pipenv.exceptions.InstallError]: ERROR: No matching distribution found for somepackage==4.0.5
ERROR: Couldn't install package: {}
 Package installation failed...
 (RequestId: 139ee786-0e73-40be-89cc-d3846bcce3d7)

This seems fairly clearly that the PIP_EXTRA_INDEX_URL is 'somehow' not on the lambda environment. as far as I can tell the only way to manipulate that environment is via pre-install scripts? I might be missing something obvious but I can't find anything in the documentation.

[kichik]

Each command is executed in a separate shell. That's why using export didn't work.

cdk-turbo-layers/src/package-python.lambda.py

Lines 89 to 93 in 5c70e78

	for command in commands:
	try:
	print(command)
	print(subprocess.check_output(command, cwd=temp, stderr=subprocess.STDOUT, shell=True, universal_newlines=True))
	except subprocess.CalledProcessError as e:

I'm not sure yet if the best solution would be to stuff all the commands together in a shell script and execute it, or to let the user specify environment variables. If we use a shell script, we lose the ability to tell the user exactly which command failed. If we allow environment variables, the user still won't be able to set dynamic environment variables (e.g. get token and set that as the password).

If you're looking for a quick solution, you might be able to write the export line into ~/.bash_profile for it to be loaded by the following commands.