Support multiroot workspaces

[jonnytest1]

i currently have multiple folders in my vscode workspace that have a .env file - icant switch the one i want because it always writes to the other one

• also weird: it overwrote the .env file with content from a compleetly different root fodler which should never happen

[EcksDy]

Hey @jonnytest, thank you for showing interest in the extension!

The extension initially wasn't intended to be used in multiroot workspaces, to not overload it with complexity from the start. The weird behaviour you experienced is the product of that, it will grab the first match and replace it.

I'm open to the possibility of supporting multiroot workspaces.

There are a few gotchas with multiroot workspaces:

• There's no way of knowing which project the user wants to modify, without prompting for it or relying on currently open file(yikes)
• Maybe the user wants to switch all of the projects to their respected preset, i.e.: pick "staging" and then have all projects attempt to find a preset named "staging"
• Detecting a multiroot workspace that doesn't have an explicit .code-workspace setup can be tricky

I'd love to hear your thoughts/ideas on how would you expect to use this feature, since I lack the experience of working with multiroot workspaces.

[jonnytest1]

• i dont think relying on currently open file to determine the current project is a bad idea neccessarily , there are lods of other (well known) extensinos that behave that way
• i think the core constraint that should be necessary is that an environment file shouldnt ever replace the content of an env in another root folder
• but if multiple environment files match the prefix its ok to replace them in each workspace with the respective env (maybe add in brackets how many files match the option in the dropdown)
• if there isnt a .code-workspace file i think its ok keep it as undefined behaviour - unless there is another way to get he current workspace roots 🤔

it might also be a good idea to allow some settings in the folder settings instead of user or workspace settings depending on wether they make sense (for example allowing a different glob pattern per folder)

[EcksDy]

• I'd love to look at examples if you can provide them :)
• This is a must, currently it behaves this way because the extension has no concept of "another root folder"
• I'm thinking that an additional step can be added before the .env select when multi root is detected, It will prompt the user to select a specific folder to switch/all folders
• I was thinking to about looking for .envs in root subfolders, if found - that subfolder is a project folder, as a fallback

I'll most likely leave granular settings out of the scope for now.

[jonnytest1]

well my idea was most language extensions have a project specific setup - but i think thats usually a byproduct of going from the current file up to the next for example package.json or tsconfig.json or similar
the one other extension i know is

https://marketplace.visualstudio.com/items?itemName=liximomo.sftp
which is an sftp client othat uplaods the entire workspace folder

anyhow if the idea is to change multiple envs with one click then its not strictly neccessary to determine the current project as long as its possible to differentiate them

[EcksDy]

I understand, took a look at the extension, thank you for your input :)
Currently thinking about the multiple .envs approach.

[natefabian18]

Would like to chime in to say I would love for this feature as its the only thing holding me back from using this extension as its exactly what I was looking for. Would love to try and support the development in whatever way I can.

[EcksDy]

Hey @natefabian18, thank you for the comment.
The higher the interest in this feature rises, the more inclined I'll be to attempt a solution.

Personally, I've been mostly working with monorepos for the past couple of years, although I don't use multiroot workspaces. The amount of approaches on how to structure and manage monorepos feels daunting.

Developing this extension, the guiding principle was that you dont need any additional files in the target repo, besides the presets, to make it work.
But whenever I think of a solution, I keep coming back to a .envswitcher JSON config file, which purpose is to list project directories and their presets.

[natefabian18]

@EcksDy Personally I tend to work in many disjointed projects that all link together for my primary work so we cant structure them as mono repos. I tend to like the suggestion made by @jonnytest1

i think the core constraint that should be necessary is that an environment file shouldnt ever replace the content of an env in another root folder

Especially with the v0.4.3 release of showing the absolute path to greatly reduce the confusion of what exactly your switching to what. I have no input on if having 2 files with the same name should switch them both, Ideally it could be left as a setting and support both.
I don't love the idea of an .envswitcher file as it would just be another config to ignore in the root of my projects. I would point to the actual .code-workspace file itself that defines how the workspace operates for dealing with the multiple roots. Maybe a list of coupled folder roots? Outside of that im fine with just running the switch command multiple times if im switching multiple roots over to new .envs since they are clearly marked by the absolute path.

[EcksDy]

@natefabian18 That constraint is a good guideline, I agree, as for the other points:

I don't love the idea of an .envswitcher file as it would just be another config to ignore in the root of my projects.

Neither do I. The repo will have to ignore it, which is not optimal, but it will allow for a more generic and inclusive solution. The more I think about it the more it makes sense, as it allows to set behavior settings that will dictate strategies of handling missing files in some of the sub-roots or how you should resolve what you want to switch: all or just current sub-root, etc.

I would point to the actual .code-workspace file itself that defines how the workspace operates for dealing with the multiple roots.

Relying on .code-workspace locks you in that particular workflow, and as I said before I've never used multi root workspaces in VSCode. That's not a workflow I'm familiar with and don't see myself switching to it unless it's day job related.
That's why I lean towards the .envswitcher solution.

Outside of that im fine with just running the switch command multiple times if im switching multiple roots over to new .envs since they are clearly marked by the absolute path.

Aha, here's the catch, currently the switch command assumes there is only one target file, so when you switch it will just show you all presets found by the glob and switch the contents of the target to the selected preset. The button in the status bar is also tied to the contents of the target file.

• What do you show on the button when there are 10 different targets?
• Does calling the switch command ask you which target to switch and then provides you the presets to pick from?
• How will it decide which presets to suggest?

With so many steps at this point I feel like the extension isn't saving that much time anymore and it does feel like multiroot/monorepo setups will require a different UI element to represent them.

[jonnytest1]

🤔 maybe a unique file in the .vscode folder is the better solution - then it can be individually .gitignored if a team decided its not shared confiugartion

[natefabian18]

@jonnytest1 I do support the dedicated file in the .vscode folder as it is meta information as to how VSCode treats the project and not specific to the project itself. The question then becomes how to manage the multiple .vscode folders across m.workspaces. Having an env-switcher.json file in the .vscode folder and merging all the definitions across the workspaces into one main object?

[jonnytest1]

🤔 yeah i tihnk merging would be ok

1. collect all env files per folder root
2. show list with all file names merged(and how many roots have one with the name , if only one show path info )
3. if one is picked go thoough all roots and check if there is an env file with the picked name and set the .env to it

[EcksDy]

I appreciate that you all are using multi-root workspaces, it introduces me to a different use case than I'm accustomed to.

Initially, I had the idea of organizing everything within one file in parent-root folder, given that I'm primarily a monorepo user. I see that it can make sense to let each sub-root define it's own presets and globs. Following that logic, the main-root would establish global behavior while sub-roots would serve as overrides.

Here's how I envision the file tree in a multi-root workspace:

📁 .vscode
└── 🛠️ .envswitcher
📁 a-project
├── 📁 .vscode
│   └── 🛠️ .envswitcher
├── 📁 envs
│   ├── ⚙️ local.env
│   └── ⚙️ staging.env
└── ⚙️ target.env
📁 b-project
├── 📁 presets
│   ├── ⚙️ local.env
│   └── ⚙️ staging.env
└── ⚙️ .env
📁 c-project
├── 📁 .vscode
│   └── 🛠️ .envswitcher
├── 📁 presets
│   ├── ⚙️ .env.local
│   └── ⚙️ .env.staging
└── ⚙️ .env
📁 documentation

The contents of the files would be as follows, starting with the main-root:

{
  "*": {
    "target": ".env",
    "targetExclude": ["**/node_modules/**"],
    "presets": "presets/*.env",
    "presetsExclude": ["**/node_modules/**"]
  },
  "exclude": ["documentation/**"]
}

a-project:

{
  ".": {
    "target": "target.env",
    "presets": "envs/*.env",
  }
}

c-project:

{
  ".": {
    "presets": "presets/.env.*",
  }
}

• The wildcard (*) in the main-root configuration applies to all directories found in the main-root.
• The exclude directive is meant to exclude directories from being treated as switchable projects.
• If no file matched the target glob, the directory won't be displayed.
• The . in sub-root configs doesn't mean the directory of the .envswitcher file(as it's in .vscode)

When sub-root configs are merged into the main-root config, the . will be replaced with the respective folder that matched the wildcard.

I think this covers the multi-root scenario.

[EcksDy]

The monorepo scenario is a little different, but the concepts remain the same: wildcards and overrides.

Let's take an NX monorepo setup as an example:

📁 .vscode
└── 🛠️ .envswitcher
📁 apps
├── 📁 a-project
│   ├── 📁 envs
│   │   ├── ⚙️ local.env
│   │   └── ⚙️ staging.env
│   └── ⚙️ target.env
├── 📁 b-project
│   ├── 📁 presets
│   │   ├── ⚙️ local.env
│   │   └── ⚙️ staging.env
│   └── ⚙️ .env
├── 📁 c-project
│   ├── 📁 presets
│   │   ├── ⚙️ .env.local
│   │   └── ⚙️ .env.staging
│   └── ⚙️ .env
└── 📁 documentation

In a monorepo, I would personally avoid cluttering the project folders, which are dedicated to code, with global tooling configurations. Instead, I'd place the configuration either in the .vscode directory or directly in the root. The extension can support both approaches.

With just one configuration file, it would appear as follows:

{
  "apps/*": {
    "target": ".env",
    "targetExclude": "**/node_modules/**",
    "presets": "presets/*.env",
    "presetsExclude": "**/node_modules/**",
    "overrides": {
      "a-project": {
        "target": "target.env",
        "presets": "envs/*.env"
      },
      "c-project": {
        "presets": "presets/.env.*"
      }
    }
  }
}

• The properties within the overrides section would impact the directories matched by the wildcard (*).

These are my thought on monorepo implementation. There may be specific edge cases that I haven't considered, but I believe this provides a solid starting point.

[natefabian18]

@EcksDy #17 (comment)
It is worth noting that in multi-root projects is its possible to have a workspace with no "Root" folder. One of the benefits of multi-root projects is that there is no root so I can pull folders from all over the system. The way I have seen other extensions handle this kinda situation is to just assume the first project is the "Main Root" and expand from there. So where as in your comment there is a root .vscode>.envswitcher it would instead use a-project>.vscode>.envswitcher

📁 a-project
├── 📁 .vscode
│   └── 🛠️ .envswitcher //This is the main file
├── 📁 envs
│   ├── ⚙️ local.env
│   └── ⚙️ staging.env
└── ⚙️ target.env
📁 b-project
├── 📁 presets
│   ├── ⚙️ local.env
│   └── ⚙️ staging.env
└── ⚙️ .env
📁 c-project
├── 📁 .vscode
│   └── 🛠️ .envswitcher //extends the main file
├── 📁 presets
│   ├── ⚙️ .env.local
│   └── ⚙️ .env.staging
└── ⚙️ .env
📁 documentation

I assume the merging would be very similar if not exactly the same just that the a-project would have both

{
  "*": {
    "target": ".env",
    "targetExclude": ["**/node_modules/**"],
    "presets": "presets/*.env",
    "presetsExclude": ["**/node_modules/**"]
  },
  "exclude": ["documentation/**"]
}

and

{
  ".": {
    "target": "target.env",
    "presets": "envs/*.env",
  }
}

Into something like

{
 "*": {
    "target": ".env",
    "targetExclude": ["**/node_modules/**"],
    "presets": "presets/*.env",
    "presetsExclude": ["**/node_modules/**"]
  },
  "exclude": ["documentation/**"]
  ".": {
    "target": "target.env",
    "presets": "envs/*.env",
  }
}

Let me know if im just not understanding something

[jonnytest1]

you can also write configs directly into the workspace config json

[EcksDy]

@natefabian18

One of the benefits of multi-root projects is that there is no root so I can pull folders from all over the system.

Good that you've brought it up, I wasn't aware that's the case.
This also complicates the matter quite a lot. So far with a regular workspace, the logic is built on top of a root folder against which everything resolves. The current logic supports monorepos as they have one root.
Something to keep in mind.

The way I have seen other extensions handle this kinda situation is to just assume the first project is the "Main Root" and expand from there.

Oooh, that's no bueno, folder order can change with every new folder added.
There's also the matter of merging such a scenario where a project takes a leading role, in your example:

{
 "*": {
    "target": ".env",
    "targetExclude": ["**/node_modules/**"],
    "presets": "presets/*.env",
    "presetsExclude": ["**/node_modules/**"]
  },
  "exclude": ["documentation/**"]
  ".": {
    "target": "target.env",
    "presets": "envs/*.env",
  }
}

* and . are meaningless in this context. They're supposed to be resolved as part of the merging, based on where the .envswitcher that had the * or . configuration is found. I also dislike that one, randomly selected, project will suddenly get a higher importance, it just feels incorrect. Merging vertically against a base(overriding), is a lot easier and clearer than trying to merge horizontally.

@jonnytest1

you can also write configs directly into the workspace config json

This can be a good solution, some questions to both of you:

• Is there always a workspace-config.json file present when working with multi-root workspaces?
• In case it's not always present, is it realistic to require adding workspace-config.json to use the extension with multi-root workspaces?
• Is it a common practice to add tooling configuration to workspace-config.json?

[natefabian18]

@EcksDy

Is there always a workspace-config.json file present when working with multi-root workspaces?

A workspace-config.json is not required for a multiroot workspace but it is to save the workspace. You can open vscode right now click add folder to workspace and you have technically created a multiroot workspace. The catch is outside of vscode remembering what you recently had open there is no way to recall this workspace. See documentation

In case it's not always present, is it realistic to require adding workspace-config.json to use the extension with multi-root workspaces?

In my opinion its a reasonable expectation to have as I dont know anyone using complex multi-root workspaces that doesn't also use workspace-config.json

Is it a common practice to add tooling configuration to workspace-config.json?

I personally havent seen tooling be added to the workspace-config.json but the documentation explicitly states that configurations that apply to the whole workspace should be placed in here then overridden with the .vscode/settings.json. As this extension is more editor tooling then specific project tooling I think its safe to put those configs in here .

[EcksDy]

@natefabian18
Thank you, I got some reading to do.
This is going to be quite a task, I'll add a TODO draft to a Planned section in README.md over the weekend,.

[natefabian18]

@EcksDy
Let me know if there is any work that can be off loaded. I know this is a lot of extra work for a workflow you dont particularly use.

[EcksDy]

Progress update

Started working on config file detection and found out that there's an issue with that, here are the thoughts on workaround:

• There is no way to locate the .code-workspace file via the extension
• M.workspaces will provide URIs to roots, these URIs are absolute paths
• Extension can write to settings on different scopes(user/workspace/m.workspace)
• Settings can be complex json objects, just not as nice to work with
• I think to just use settings for the workspace config(single/monorepo)
  • Fallback to a .envswitcher in root or .vscode folders(optional)
• When a m.workspace detected, extension will read from the m.workspace settings to see if anything was configured, then prepopulate the settings with absolute paths of the folders if settings were empty
• There's an event of added/removed folder in m.workspaces, extension will be able to sync the settings accordingly

[jonnytest1]

writing to workspace settings should be the same as writing to .code-workspace (if the workspace is saved)

[EcksDy]

Yeah, that's correct.

I remembered why initially I was trying to avoid writing to .vscode/settings.json and its counterparts. It's because that file might be committed and then someones specific setup cannot be gitignored.
I think this will be a good motivation to add a fallback option to use .envswitcher as primary source of config if found.

I'm currently working on adapting the config module to be workspace aware.

[jonnytest1]

personally i think committing the .vscode/settings.json is precicely the advantage ^^ , if you have a team setup where everyone uses the same envs then comitting it is probably preferable (only works if all paths are relative tho )
so imo the best way is to allow both files and then merging the configs

[EcksDy]

I didn't find anyway to access the relative paths in the "folders" directive in .code-workspace file.
I can only get the list of open folders in the m.workspace as absolute paths. Couldn't find any way of locating the .code-workspace or getting workspace paths as relative to .code-workspace file.
Are you familiar with any hack or workaround? I've tried this but it just return absolute path:

    workspace.workspaceFolders?.forEach((folder) => {
      console.log(workspace.asRelativePath(folder.uri));
    });

[jonnytest1]

is sthere a way to get the workspace root or code-workspace file ?
i dont think the hack i have is in any way recommended 😅

[EcksDy]

You have workspace roots(m.workspaces), but they're just the absolute paths to open folders. There is no way that I know of to get the .code-workspace file or its path.
I'm thinking about prompting the user on first activation to point to that file.

Having absolute paths in .code-workspace is not an issue as long as it's not committed. But if the file is committed, then it must have relative paths.

I guess I can prompt the user to provide the path to the file on first activation if I find that the paths specified there aren't absolute and then save it(.code-workspace path) in the extension state along with resolved folders paths.
This will require checking that the folders exist where I expect them to, essentially checking that .code-workspace wasn't moved.

This is the best solution I can come up with so far.

[jonnytest1]

function parseJsonWithComments<T = any>(json: string): T {
    const withoutComments = json.split('\n')
        .filter(line => !line.trim().startsWith('//'))
        .join('\n')

        /**
         * replace floating commas
         */
        .replace(/},([\s]*)}/m, '}$1}');

    return JSON.parse(withoutComments) as T;
}

function getWorkspaceConfigs() {
  switch (process.platform) {
    case 'win32':
      return `${process.env.APPDATA}\\Code\\User\\workspaceStorage`;
    case 'linux':
      return `${process.env.HOME}/.config/Code/User/workspaceStorage`;
    case 'darwin':
      return `${process.env.HOME}/Library/Application Support/Code/User/workspaceStorage`;
    default:
      return `${process.env.HOME}/.config/Code/User/workspaceStorage`;
  }
}
 const workspaceConfigsPath = getWorkspaceConfigs();

  if (!existsSync(workspaceConfigsPath)) {
    return;
  }
  const files = await promises.readdir(workspaceConfigsPath);

  let latest;
  let latestTs: Date | null = null;

  for (const file of files) {
    try {
      const stats = await promises.stat(join(workspaceConfigsPath, file, 'state.vscdb'));
      if (!latestTs || latestTs < stats.mtime) {
        latestTs = stats.mtime;
        latest = file;
      }
    } catch (e) {
      if (e.message.includes('no such file')) {
        continue;
      }
      throw e;
    }
  }
  const workspaceJsonFile = join(workspaceConfigsPath, latest, 'workspace.json');
  if (!existsSync(workspaceJsonFile)) {
    return;
  }
  const workspaceJsonStr = await promises.readFile(workspaceJsonFile, {
    encoding: 'utf8',
  });
  const workspaceJson = parseJsonWithComments(workspaceJsonStr);

🤔 wasnt even as bad as i rember - i was almost certain it was by reading the process args of the currently running vscode processes 😅

[jonnytest1]

🤔 i dont think we need the relative paths though , all the relative paths we use should be relative to the corresponding workspace root (or its ok for them to be absolute ) or am i forgetting something ?

[EcksDy]

Where did you get this snippet from? That's some deep VSCode black magic! :)

Unfortunately in my testing of it, it's picking up the wrong workspace.json. So we can't rely on it. What if the user saved the m.workspace somewhere else on the machine manually through Save workspace as...?

• I'm unfamiliar with m.workspaces setups, so I'm not sure if people committing the .code-workspace?
• It makes sense to me that they might want to do that
  • In that case extension should be able to work with relative paths in config
  • We don't know where the .code-workspace file is saved
  • There's no starting point for the relative paths to resolve

Part A

Use absolute paths in config by default:

• This will work for most users who don't commit their .code-workspace
• Users who commit the file will probably change it to relative path

Part B

When extension detects relative path in config:

• User is prompted to locate the path of the .code-workspace
• Absolute path to .code-workspace is saved to persistent extension state
• Relative paths in config are resolved against that absolute path and saved to persistent state
• On extension activation all of the paths are stated to make sure nothing moved
• If .code-workspace not found
  • Prompt for it again
  • Reset all paths
• If any of the folders not found
  • Read the folders directive in .code-workspace
  • Update config accordingly

It's not clean and I'm sure there will be edge cases, but this should be foolproof for 99% of the use cases.

[EcksDy]

I've re-read what you said, and you might be onto something...
In m.workspaces I get a list of absolute paths for the workspace roots:

• /home/user/projects/project-a
• /home/user/projects/nested/project-b
• /home/user/somewhere-else/project-c
• /home/user/somewhere-else/project-d

We can take an approach where configs look like:

• */project-a
• */project-b
• */project-c
• */project-d

Then match the directory base name from the workspace roots against the config.

Problem is that nothing prevents you from having 2 directories with the same name and then it falls apart:

• /home/user/projects/project-a
• /home/user/projects/nested/project-a
• /home/user/somewhere-else/project-a
• /home/user/somewhere-else/work/project-a

We can try recursively go up a level until uniqueness is achieved, but how will you know when to stop and which of the directories will get the uniqueness treatment?

How likely is it even to have 2 same named folders in m.workspaces? Nothing prevents it, but how likely is it? Maybe it affects a small enough group that can be politely asked to rename their directories to be able to use the tool?

Edit

I've made this github search and looked at a few pages. Didn't find any file with duplicate directory names.
I did find some unexpected setups but they didn't seem relevant as they manage packages not services.

[natefabian18]

To once again add my two cents. Normally I havent seen the code-workspace commited as with relative paths everything would be bundled together and your might as well have a monorepo at that point. I dont know why you would ever want two folders named the same in one workspace as it would just making features of multiroot workspaces confusing to use.

[jonnytest1]

oh 🤔i think vscode may have changed the name format - instead of a timestamp it is now a (semi)random string
it should still work if instead of using the name as a timestamp we just check when the folder was created though its metadata
(i worked that snipped out myself ^^)

[EcksDy]

Progress update

• Found a useful mini framework for m.workspaces support(vscode-helpers)
• Naively converted the extension to use said framework
• Architecture is not aware of multiple roots, hardcoded the first root
• There's some issue with dependencies of vscode-helpers opened a PR on their repo

Next steps would be:

• Start adapting the architecture to multiple roots
• Maybe switch to event emitter for some actions
• Start merging with the UI effort in the other branch
• Tests, tests, tests

[EcksDy]

Progress update

• The extension is almost ready for m.workspaces, the ground work is done
• Trying out event emitter to decouple different parts from each other
• Converted webview to Svelte
• Currently webview displays and controls one workspace(not very useful)

Next steps:

• Figure out the rest of the m.workspaces support and undo the hardcoding to one workspace
• Fix ci/cd and branching issues I have
• Implement the locking mechanism to lock specific projects from changing