Adding Support for Native Windows (Lock Configurations)

[hrishikeshh]

Is your feature request related to a problem? Please describe.
Yes. This feature request is based upon a bottle neck faced while dev environment setup by multiple windows users.

Whenever windows users are trying to setup their dev environment using command make install-pudl, it leads to platform validation issues since environment configurations (conda.yml file) only covers locks for:-

• linux-64,
• osx-64,
• osx-arm64

Refer the specific configs in the conda-lock.yml

metadata:
  content_hash:
    linux-64: 09c78383a83af1c2bf06013b06ef870afea96baa86bd8d9373ec0d5215804c8d
    osx-64: 7e63fa499a48eca9f5a939548fc6b8f2b84d82642073a139604fe68ef780e1a2
    osx-arm64: c694e1530235797a6bda755e6e04d7577cca6ca86ca8e51f52e72f6c85b73f8a
  channels:
    - url: conda-forge
      used_env_vars: []
    - url: defaults
      used_env_vars: []
  platforms:
    - linux-64
    - osx-64
    - osx-arm64
  sources:
    - ../pyproject.toml

This results into PlatformValidationError.

There is no directly available solution for native window systems using powershell or cmd

Describe the solution you'd like

Support for windows (win64) should be directly added in the environment. A section in conda-lock.yml or a seperate config for win64 should be there if technically feasible.

Describe alternatives you've considered

• Currently to bypass this, users have to use either Windows Subsystem for Linux wsl or Linux Ubuntu.
• Alternatively, can add the following suitable arguments to the conda-lock command:
  --platform=win-64
--platform=linux-64
--platform=osx-64
--platform=osx-arm64

[zaneselvans]

Hi @hrishikeshh, have you tried building the environment with the windows platform enabled? Did it work? When I initially set up the conda-lock configuration I couldn't get it to solve the environment, as there were some required packages that didn't exist on Windows.

[hrishikeshh]

Hi @zaneselvans, Thanks for prompt reply.

Although conda-lock.yml explicitly mentions not to edit but I tried building the environment with win-64 windows platform enabled. It didn't work.

I have tried via miniforge console as well as with wsl console.

Pre-enabling

Post-enabling

The existing "Getting Started Documentation" needs more context for Windows user mentioning possible pitfalls especially since windows user have only option of using Windows Subsystem for Linux with existing locking policies.

Once we find a solution or way around, I will updated existing docs with more detailed and specific instructions for windows dev support.

[zaneselvans]

To change the conda-lock configuration to test inclusion of the Windows platform you'll need to edit the conda-lock section of pyproject.toml and then attempt to re-lock the dependencies with:

make conda-clean conda-lock.yml

At the moment, development / running the PUDL ETL is not supported on Windows, partly because of these environment issues. It needs to be run on a Unix system (which includes MacOS). So we recommend that Windows users use the built-in Windows Subsystem for Linux (WSL).

Edit: Having just tried this, it seems that the blocker is (still) google-cloud-sdk which is only available for linux-64, osx-64 and osx-arm64, and good luck getting The Google to change its ways!

I will add a comment in pyproject.toml noting this for future reference.