[BUG] Get wrong tcl on Python 3.10 or lator with Windows

[mikeda-nagano]

I found code seems to cause of this problem.
but I hesitate to commit code because I don't understand whole VUnit system,
and this is my first contribution for OSS.
so, I created bug report for someone else to fix this.

thank you.

Details

I put wave.do, the waveform window configure script of ModelSim on same directory of testbench.
and set vunit option modelsim.init_file.gui to $vunit_tb_path/wave.do in run.py.

then run test with CLI option -g | --gui, fail to execute script after GUI launch.
error message says script file C:/ProjectDirectory/C:/ProjectDirectory/tb/wave.do not found.
ofcourse, this is wrong path because of repeating C:/ProjectDirectory twice.

this problem only occurs Python 3.10 or later with Windows, but successed on 3.9.
(I don't know result with Linux)

Environment

• OS: Windows 11
• Python: CPython 64bit Windows (3.10 or lator)
• vunit: 4.7.0

Reproduction

install CPython 3.9 and 3.10 to your Windows PC.

clone vunit repository on C:/WorkingDirectory, and change current directory to root of vunit repo.

PS C:\WorkingDirectory> git clone <url of vunit repo>
PS C:\WorkingDirectory> cd vunit
PS C:\WorkingDirectory\vunit>

create python code sample.py in current directory.

# sample.py

from vunit.sim_if import vsim_simulator_mixin as mixin

class config_dummy:
    design_unit_name = "sample_design"
    tb_path = "C:/WorkingDirectory/tb"

tcl = mixin.VsimSimulatorMixin._source_tcl_file(
        file_name="$vunit_tb_path/wave.do",
        config=config_dummy(),
        message="sample message")

print(tcl)

run sample.py with option to select Python version.

# Python 3.9
PS C:\WorkingDirectory\vunit> py -3.9 sample.py > 3_9.tcl

# or Python 3.10
PS C:\WorkingDirectory\vunit> py -3.10 sample.py > 3_10.tcl

here is diff of results.
- is 3.9, + is 3.10.

    set vunit_tb_path "C:/WorkingDirectory/tb"
    set vunit_tb_name "sample"
-   set file_name "$vunit_tb_path/wave.do"
+   set file_name "C:/WorkingDirectory/vunit/$vunit_tb_path/wave.do"
    puts "Sourcing file ${file_name} from some message"
    if {[catch {source ${file_name}} error_msg]} {
        puts "Sourcing ${file_name} failed"
        puts ${error_msg}
        return true
    }

wrong tcl created on Python 3.10.
tcl variable ${file_name} will evaluate as C:/WorkingDirectory/vunit/C:/WorkingDirectory/tb/wave.do.

Cause of Problem

I think the cause is a change of behavior pathlib.Path.resolve().

# on 3.9

>>> import pathlib
>>> pathlib.Path("$vunit_path/wave.do").resolve()
WindowsPath('$vunit_path/wave.do')

# on 3.10
# (current directory is C:/WorkingDirectory)

>>> import pathlib
>>> pathlib.Path("$vunit_path/wave.do").resolve()
WindowsPath('C:/WorkingDirectory/$vunit_path/wave.do')

so VUnit generates wrong tcl on static method _source_tcl_file() in vunit/sim_if/vsim_simulator_mixin.py.

# vsim_simulator_mixin.py

    @staticmethod
    def _source_tcl_file(file_name, config, message):
        """
        Create TCL to source a file and catch errors
        Also defines the vunit_tb_path variable as the config.tb_path
        and the vunit_tb_name variable as the config.design_unit_name

        """
        template = """
    set vunit_tb_path "%s"
    set vunit_tb_name "%s"
    set file_name "%s"
    puts "Sourcing file ${file_name} from %s"
    if {[catch {source ${file_name}} error_msg]} {
        puts "Sourcing ${file_name} failed"
        puts ${error_msg}
        return true
    }
"""
        tcl = template % (
            fix_path(str(Path(config.tb_path).resolve())),
            config.design_unit_name,
            fix_path(str(Path(file_name).resolve())),
            message,
        )
        return tcl

I think it will fix problem by removing resolve(),
but can't be sure there are no side effects.

tcl = template % (
    fix_path(str(Path(config.tb_path).resolve())),
        config.design_unit_name,
-       fix_path(str(Path(file_name).resolve())),
+       fix_path(str(Path(file_name))),
        message,
    )

P.S.

it might to occurs same problem for Actvie HDL.
see _create_user_init_function() in vunit/sim_if/activehdl.py.

tcl += f'source "{fix_path(str(Path(init_file).resolve()))!s}"\n' looks like occurs same problem,
if there are individual init_file for each testbenchs.
(I'm not Active HDL user, so this may be incorrect)

# activehdl.py

    def _create_user_init_function(self, config):
        """
        Create the vunit_user_init function which sources the user defined TCL file in
        simulator_name.init_file.gui.
        Also defines the vunit_tb_path and the vunit_tb_name variable.
        """
        opt_name = self.name + ".init_file.gui"
        init_file = config.sim_options.get(opt_name, None)
        tcl = "proc vunit_user_init {} {\n"
        if init_file is not None:
            tcl += f"set vunit_tb_name {config.design_unit_name}\n"
            tcl += f"set vunit_tb_path {fix_path(str(Path(config.tb_path).resolve()))}\n"
            tcl += f'source "{fix_path(str(Path(init_file).resolve()))!s}"\n'
        tcl += "    return 0\n"
        tcl += "}\n"
        return tcl

[LarsAsplund]

The vunit_tb_path variable was introduced to enable referencing the testbench directory from within a sourced script, not to use it when specifying the script path. However, what we can consider is to skip resolve() in certain situations, for example when the file name starts with $vunit_tb_path. I will give it some thought to see if there are unwanted side-effects.

[LarsAsplund]

The only problem I see is that $ is a valid file path character. For that reason, I would limit such an exception to paths starting with $vunit_tb_path