toggle ignore_subdirs by workspace

[avsm]

When experimenting with vendoring dependencies in Dune, I noticed that it would be useful to be able to toggle the use of vendored files at build-time. For example, if there is a directory called vendor/ in a repository, then it could be compiled in a few scenarios:

• via opam, where the vendor/ directory should be ignored since the dependencies can be satisfied by opam
• a direct git clone, where we want to use the vendored files as there is no opam environment.
• when updating the vendor universe, its useful to disable the current set. This came up recently since I couldn't build something with 4.07 until I upgraded the vendored version of Base.

The workspace feature seems like a pretty good fit for this, but we need to extend it to add the facility to use the ignore_subdir command that is in a dune file at the moment.

A workaround at the moment is to use an OCaml-generated dune file and test for an environment variable or similar to include the clause, but I'd like to avoid that if possible.

[rgrinberg]

@diml @emillon what do you think about this feature? I don't think there's anything wrong with it in principle, but it's kind of a huge gun for a small problem. Do we have any other uses for varying views of the file system per context?

[emillon]

Would a command line flag work? we already have flags to control how promotion works for example.

[rgrinberg]

I think adding a command line option for often used configurations is a good idea, but I dislike having cli only configuration for most options. There are some special cases where a cli only option make sense, but this doesn't seem like one of them.

[rgrinberg]

Which command line option did you have in mind btw? A way to toggle to ignored sub directories in the build?

[ghost]

If it's an option that you need to pass to every invocation (build, runtest, ...) then an option in the workspace configuration file seems good.

I admit that I don't fully understand the use case for this feature. Is it basically like an optimization? i.e. you already installed the dependencies via opam so you don't want to recompile them again?

[emillon]

Which command line option did you have in mind btw?

I haven't thought this through, but yes I was thinking something like --ignored-dir vendor or something.

[avsm]

@dim, the motivation is due to needing the flexibility in cases like this:

when updating the vendor universe, its useful to disable the current set. This came up recently since I couldn't build something with 4.07 until I upgraded the vendored version of Base.

In this case, I had to rm -rf the vendor directory and revendor it from scratch. Not too bad, but it would have been convenient to be able to build the local tree without including vendor/.

Also, it would be useful to have a workspace that uses the opam libs, and one which uses the locally vendored libs, to test that both work.

[ghost]

Thanks, I understand better why we need this. Having it in the workspace file seems fine.