Concurrency based on coroutines is making use of cooperative multitasking, leaving the concurrent execution of tasks up to the runtime. To elaborate a bit on what that implies, let me just ask you the following question: Is there something less cooperative than a task that doesn't yield its control back to the main thread?
Regarding the fire-and-forget pattern I can think of at least two issues, let me illustrate them with the following example:
import asyncio
from typing import Any, Coroutine
_background_tasks = set[asyncio.Task[None]]()
def _fire_and_forget(coro: Coroutine[Any, Any, None]) -> None:
task = asyncio.create_task(coro)
_background_tasks.add(task)
task.add_done_callback(_background_tasks.discard)
async def _run_a() -> None: # to illustrate issue A
raise RuntimeError()
async def _run_b() -> None: # to illustrate issue B
await asyncio.sleep(1)
print('done')
async def main() -> None:
# issue A: Exceptions can't be caught
try:
_fire_and_forget(_run_a())
except RuntimeError as exc:
print(f'Exception caught: {exc}')
# issue B: Task won't complete
_fire_and_forget(_run_b())
if __name__ == '__main__':
asyncio.run(main())
Feel free to comment out either task in the main function to observe the resulting behaviors individually.
For issue A: Any raised error in the background task can't be caught and will crash the running main thread (the process)
For issue B: Background tasks won't be completed if the main thread comes to a halt
With the decision for the fire-and-forget pattern you'll make a deliberate choice to leave any control of the runtime up to blind chance. So from an engineering POV that pattern isn't a solution-pattern to some real problem, it's rather a problem-pattern that demands a reworked solution.
Losing control of a background task (and therefor the runtime) might be fine for some demo project, but I think you'll want to notice raised errors in any serious production system, especially for any work that takes several minutes to complete.
> Is there something less cooperative than a task that doesn't yield its control back to the main thread?
Of course it does yield back to the main thread in my example, at each await point, just like any other cooperative task.
In my case, I specifically want an independent execution of a task. Admittedly, it has to catch its own exceptions and deal with them, as you pointed out, because that's part of being independent.
(Technically, in issue A it doesn't crash the running thread. The event loop catches the exception, but it complains later when the task is garbage collected. Issue B is fine for my use - when the event loop shuts down, it cancels remaining tasks, which is exactly right for my server.)
Regarding the fire-and-forget pattern I can think of at least two issues, let me illustrate them with the following example:
Feel free to comment out either task in the main function to observe the resulting behaviors individually.For issue A: Any raised error in the background task can't be caught and will crash the running main thread (the process)
For issue B: Background tasks won't be completed if the main thread comes to a halt
With the decision for the fire-and-forget pattern you'll make a deliberate choice to leave any control of the runtime up to blind chance. So from an engineering POV that pattern isn't a solution-pattern to some real problem, it's rather a problem-pattern that demands a reworked solution.
> How do you want it to be structured
Take a look at the caveats for FastAPI/Starlette Background Tasks: https://fastapi.tiangolo.com/tutorial/background-tasks/#cave...
Losing control of a background task (and therefor the runtime) might be fine for some demo project, but I think you'll want to notice raised errors in any serious production system, especially for any work that takes several minutes to complete.