While running multiple independent tasks in parallel which in fact is the primary function of map_task
The code that caused the error is as follows:
from flytekit import map_task, task, workflow
@task
def do_something(value: str) -> str:
print(f"launched: {value}", flush=True)
time.sleep(60) # fakes long process time
return f"{value}-processed"
@workflow
def do_multiple_things() -> list[str]:
values = ["foo", "bar", "baz"]
return map_task(do_something)(value=values)
So here is the solution:
Local runs will not do parallel just yet. (Making flyte-kit execute local runs in parallel is part of a broader project that flyte has plans for someday, but no definite timeline).
Top comments (1)
Hi
Could you add an intro to this post? I'm not sure I follow what a map task is and what this problem is all about?