[BUG] Getting Started code errors out on MacOS M1

[juan-park]

Describe the bug

I tried running the following code locally:

import pandas as pd
from sklearn.datasets import load_wine
from sklearn.linear_model import LogisticRegression

import flytekit.extras.sklearn
from flytekit import task, workflow


@task
def get_data() -> pd.DataFrame:
    """Get the wine dataset."""
    return load_wine(as_frame=True).frame

@task
def process_data(data: pd.DataFrame) -> pd.DataFrame:
    """Simplify the task from a 3-class to a binary classification problem."""
    return data.assign(target=lambda x: x["target"].where(x["target"] == 0, 1))

@task
def train_model(data: pd.DataFrame, hyperparameters: dict) -> LogisticRegression:
    """Train a model on the wine dataset."""
    features = data.drop("target", axis="columns")
    target = data["target"]
    return LogisticRegression(max_iter=3000, **hyperparameters).fit(features, target)

@workflow
def training_workflow(hyperparameters: dict) -> LogisticRegression:
    """Put all of the steps together into a single workflow."""
    data = get_data()
    processed_data = process_data(data=data)
    return train_model(
        data=processed_data,
        hyperparameters=hyperparameters,
    )

Using command: pyflyte run example.py training_workflow --hyperparameters '{"C": 0.1}'

And i get the following bug:

I don't think it's necessarily related to M1, as I tried switching arch to x86_64 and rerunning it and I get the same error.

During the run, I think it tries to save the pandas dataframe, but fails. Here is the output of the script (empty folders, note the file://:

Expected behavior

Code should run to completion

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[welcome]

Thank you for opening your first issue here! 🛠

[juan-park]

Specifically, I'm on
Python 3.11.6 (main, Nov 11 2023, 21:30:07) [Clang 15.0.0 (clang-1500.0.40.1)] on darwin
Flytekit version 1.10.0 (also tried 1.8.0)

[eapolinario]

@juan-park , this was fixed with the latest flytekit release (1.10.1). Can you give it a try?

[juan-park]

Hey @eapolinario, tested it on 1.10.2 and it works, thanks 🙏!

[eapolinario]

This was fixed in flytekit 1.10.2.