At our company we use Redis, it's lightweight and has list structure which can be used as a push/pop queue
The basic gist of it is that on one end a producer pushes to the list and a consumer(s) on the other end pops the job and executes it. Fire and forget style.
Yeah, redis or similar are generally good and battle tested solutions. Celery and Huey use redis, as do many other task queue and scheduling solutions. The issue is then that you have to manage an additional deployment of redis. That’s not so bad if it’s a core part of your application esp if you’re hosting on a paas or major cloud, but it can make local dev and testing suck a bit more.
Yes it's true, we already use Redis for several workloads and operations so it's not really an extra thing to manage, on the contrary we don't need to learn about say rabbitmq and manage it, just worry about Redis.
Why do you consider it bad for local dev? A Redis instance literally takes 1MB memory when started.
The basic gist of it is that on one end a producer pushes to the list and a consumer(s) on the other end pops the job and executes it. Fire and forget style.