[feature] Pydantic

[mikeedjones]

Feature Area

What feature would you like to see?

I would like to be able to typehint containerized and lightweight python components, and pipelines, using Pydantic Models - and have the validation of those models occur automatically, making the validated models available inside the component code.

from pydantic import BaseModel

class MyModel(BaseModel):
    foo: str

@component(target_image="myimage")
def my_function_one(my_data: MyModel):
    print(my_data.foo)

What is the use case or pain point?

At the moment I'm passing complex hierarchical dictionaries as parameters to kubeflow using Dict[str, Any] and then validating those dictionaries inside the function code, as below

from pydantic import BaseModel

class MyModel(BaseModel):
    foo: str

@component(target_image="myimage")
def my_function_one(my_data: Dict[str, Any]):
    from . import MyModel
    my_data_validated = MyModel.model_validate(my_data)
    print(my_data_validated.foo)

Where myimage has the base model available.

Is there a workaround currently?

This gist shows my current attempt at a reusable workaround.

---

Love this idea? Give it a 👍.

[rimolive]

Please join our next Pipelines WG meeting to discuss about this feature.

[caroline-paul99]

Hi @mikeedjones Im coming from Issue #10210 what is the state of this issue? Also what version of KFP are you trying to update?

[mikeedjones]

Think mine is more of a feature request for future versions of 2.x - I'm not sure that your request is covered by what I had in mind :(

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[github-actions]

This issue has been automatically closed because it has not had recent activity. Please comment "/reopen" to reopen it.

[AndresPrez]

/reopen

[google-oss-prow]

@AndresPrez: You can't reopen an issue/PR unless you authored it or you are a collaborator.

In response to this:

/reopen

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.