snowfakery task fails on Linux when renaming working directory

[schenkkp]

Describe the bug

An error occurs when running the snowfakery task, specifically when attempting to move a directory (run_dir/template_1). The error message indicates that the directory is not empty, causing the task to fail.

Expected Behavior

The snowfakery task should complete without issues, moving or handling directories as needed. The directory should be managed appropriately even if it is not empty. Note: The command runs successfully when running locally on macOS.

Actual Behavior

On Linux, the task fails with an error [Errno 39] Directory not empty when attempting to move run_dir/template_1.

File "/usr/local/lib/python3.11/dist-packages/cumulusci/tasks/bulkdata/snowfakery.py", line 586, in _generate_and_load_initial_batch
shutil.move(template_dir, new_template_dir)
File "/usr/lib/python3.11/shutil.py", line 815, in move
os.rename(src, dst)
OSError: [Errno 39] Directory not empty: 'run_dir/template_1' -> 'run_dir/template_1'

Potential Root Cause

The snowfakery task appears to be renaming the template_1 directory to the same name. When using Python's shutil.move() to move a file or directory to the same path, an error can occur because Unix-like systems enforce logical constraints on file operations. Specifically, moving a file to the same location is effectively a no-op and doesn't make sense regarding file operations.

On macOS:

macOS handles this operation without an error, possibly because it internally ignores attempts to move a directory to itself.

On UNIX (Linux):

The operation fails with a Directory not empty error because the system sees it as an attempt to overwrite a directory (template_dir) with itself, which is not allowed when the destination directory already contains files or subdirectories.

Potential Fix

Update the _generate_and_load_initial_batch function that "rename[s] directory to reflect real number of sets created" starting at line 576 in cumulusci/tasks/bulkdata/snowfakery.py. Check if new_template_dir matches template_dir before processing shutil.move(template_dir, new_template_dir) to skip the command altogether.

Reproduction steps

1. Set up a Snowfakery recipe (recipes/delivery.yml) with a working directory (run_dir) and required Salesforce org.
2. Run the command from a Linux OS:

   cci task run snowfakery --recipe recipes/delivery.yml --org target-org --working-directory run_dir

3. Observe the error during the execution of the snowfakery task when moving run_dir/template_1.

Your CumulusCI and Python versions

CumulusCI version: 4.0.1 (/usr/local/bin/cci)
Python version: 3.11.2 (/usr/bin/python3)

Operating System

Linux 4.4.0 #1 SMP Sun Jan 10 15:06:54 PST 2016 x86_64 GNU/Linux

Windows environment

No response

CumulusCI installation method

pip

Error Gist

https://gist.github.com/schenkkp/f8f74a3a774adc76886d5908438fadee

Additional information

No response