Update how to get pipelines for new kedro version #53
Conversation
|
@cshaley do we need a little clean up here - to get the workflow matrix matching the setup.py... maybe @elanqo can help here. Not sure if github actions will take care of it.. Line 34 in 3788eba python_requires=">=3.8, <3.11", kedro-serving/.github/workflows/test.yml Line 17 in 3788eba python-version: [3.6, 3.7] python-version: [3.7] |
| @@ -5,6 +5,7 @@ | |||
| import numpy as np | |||
| import pandas as pd | |||
| from fastapi import FastAPI | |||
| from kedro.framework.project import find_pipelines | |||
There was a problem hiding this comment.
@cshaley I think the current modification is not the right one. The find_pipelines() function looks for create_pipeline`() functions in the project template, but any pipeline filtered / modified /created in the pipeline.py will not be loaded. I think the right way to load kedro pipelines is now :
from kedro.framework.project import pipelines| @@ -21,6 +21,7 @@ def serving_commands(): | |||
| @click.option( | |||
| "--pipeline", | |||
| "-n", # n for "name" | |||
| default="__default__", | |||
There was a problem hiding this comment.
Actually I did not put a default value for the pipeline argument of the serve command on purpose. I think many pipelines (including the default one) cannot be served with kedro-serving because they don't have a single input. In most ML projects, the __default__ pipeline is a combination of training + predicting + computing some metrics, and there are very often other inputs than only the raw data. in your projects, do you often most times that the __default__ can be served?
|
@rxm7706 is right, the worklflow is quite old. I've updated it in kedro-pandera recently, we can likely copy-paste it but it can be another PR. @cshaley : See my comments directly in the code |
Description
Why was this PR created?
context.pipelines does not work in kedro 0.18
Development notes
What have you changed, and how has this been tested?
changed to use new find_pipelines function, tested locally
Checklist
CHANGELOG.mdfile. Please respect Keep a Changelog guidelines.Notice
I acknowledge and agree that, by checking this box and clicking "Submit Pull Request":
I submit this contribution under the Apache 2.0 license and represent that I am entitled to do so on behalf of myself, my employer, or relevant third parties, as applicable.
I certify that (a) this contribution is my original creation and / or (b) to the extent it is not my original creation, I am authorised to submit this contribution on behalf of the original creator(s) or their licensees.
I certify that the use of this contribution as authorised by the Apache 2.0 license does not violate the intellectual property rights of anyone else.