Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

What's the actual difference between a job queue and a message queue filled with job IDs?


In a decent job queue you know when jobs get completed, whether the worker died, and whether they're otherwise stuck.


Message queues achieving the rate you said are typically not durable.


The difference is nothing. A message can be a serialized object representing a job. Or the message can be a jobId that points to a record in a db.


When I used PG for my job queue, I was already using Kafka instead, to handle immediate job entries. The reason I needed PG was for jobs scheduled at a future time.

I had no difficulty with long-running jobs because servicing jobs out of PG was simply a matter of pushing them onto the Kafka queue for immediate uptake there.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: