Tasks returning numpy array / Passing numpy arrays between tasks

[sakthi-tech4th]

I'm building a ML training workflow and my tasks return numpy arrays. When I try to run them, I get the following error message:

I also tried to override the 'allow_pickle' parameter to True. Then it started asking for the path

My command (since i'm testing)
python flyte_pipeline.py
As the guide suggests.

My questions are:

1. Does Flyte support tasks returning Numpy arrays?
2. Does Flyte support passing numpy arrays between tasks?
3. If I want to pass Numpy Arrays between the tasks, what is the best way to do it? Do you have any reference documentations?

Thanks and Regards,
Sakthi

[samhita-alla]

@sakthi-tech4th, NumPy array can be passed among the Flyte tasks. Yours is a NumPy array with dtype set to object and hence it's failing. It works with all the other dtypes. I'll add support to be able to serialize and deserialize object NumPy arrays when a user wants to pickle the data. Please bear with me while I work on the implementation.

[samhita-alla]

Created a PR: flyteorg/flytekit#1209