@pypi/@conda parsers ...

[savingoyal]

... for requirements.txt, conda.yaml, pyproject.toml

from metaflow import Config, pypi, FlowSpec, requirements_txt_parser, step

class MyFlow(FlowSpec):
    reqs = Config("reqs", default="requirements.txt", parser=requirements_txt_parser)

    @pypi(**reqs)
    @step
    def start(self):
        self.next(self.end)

    @step
    def end(self):
        pass

if __name__ == "__main__":
    MyFlow()

[saikonen]

stacked PR Introduces a MetaflowException type exception for these instead, in order to get cleaner formatting.

Is there any specific reason to stick with ValueError? from what I can tell the exception bubbles all the way and is not caught, leading to a full stack trace in stdout

[saikonen]

the previous PR had more specific line handling for requirements.txt content, specifically errors for options:
https://pip.pypa.io/en/stable/reference/requirements-file-format/#global-options

should we reintroduce these, or do we want to keep this implementation as simple as possible, and simply error out on 'unparseable' lines in the file? right now the parser throws an exception if the requirements.txt contains any options, be it global or per-requirement

[saikonen]

parser breaks with dependency list where there is whitespace between pkg and version, e.g.

dependencies:
  - python = 3.10.*