Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG]: map_task with timeout on tasks fails with min_success_ratio set #3875

Open
2 tasks done
ljstrnadiii opened this issue Jul 18, 2023 · 2 comments
Open
2 tasks done
Labels
array-node backlogged For internal use. Reserved for contributor team workflow. bug Something isn't working exo fit-and-finish

Comments

@ljstrnadiii
Copy link

Describe the bug

A failing task due to timeout should get filtered by map_task's min_success_ratio functionality. Note: timeout seems to be ignored locally and this only fails after registering.

The issue I see in the flyteui: [1/1] currentAttempt done. Last Error: USER::task execution timeout [10s] expired

An mre/snippet:

import time
from flytekit import task, workflow, map_task

@task(timeout=10)
def unpredictable_task(t: int):
    time.sleep(t)

@workflow
def unpredictable_workflow():
    result = map_task(unpredictable_task, min_success_ratio=0.0)(t=[1,2,3,11,12,13])

Expected behavior

I would expect map_task to succeed and for the result to be of length 3 corresponding to the successful tasks.

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

  • Yes

Have you read the Code of Conduct?

  • Yes
@ljstrnadiii ljstrnadiii added bug Something isn't working untriaged This issues has not yet been looked at by the Maintainers labels Jul 18, 2023
@welcome
Copy link

welcome bot commented Jul 18, 2023

Thank you for opening your first issue here! 🛠

@eapolinario eapolinario added array-node and removed untriaged This issues has not yet been looked at by the Maintainers labels Jul 21, 2023
@rnatour
Copy link

rnatour commented Sep 28, 2023

I'm noticing a related bug: it appears that the timeout applies to the entire map_task node, rather than individual tasks. I would expect that the timeout should apply to each task individually.

Going off the parent example:

import time
from flytekit import task, workflow, map_task

@task(timeout=10)
def unpredictable_task(t: int):
    time.sleep(t)

@workflow
def unpredictable_workflow():
    result = map_task(unpredictable_task, concurrency=1)(t=[1,2,3,4,5])

This causes the map_task to fail, since the entire set of tasks takes longer than 10s when they are run sequentially.

@hamersaw hamersaw added exo backlogged For internal use. Reserved for contributor team workflow. labels Nov 8, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
array-node backlogged For internal use. Reserved for contributor team workflow. bug Something isn't working exo fit-and-finish
Projects
None yet
Development

No branches or pull requests

4 participants