How to do distributed cronjobs with worker queues?

• cron

  19 12,638 0.0 Go

  a cron library for go

I had a similar design challenge recently and I haven’t worked it out completely yet. But I needed to dynamically create CronJob a based on a request and run each job in a go routine. I landed on https://github.com/robfig/cron which supports that behavior and removing jobs as well. I see this as being similar to your need since each scheduled cron entry via this library is sorta like having a queue with each having their own possible schedule. But I suppose it’s totally different from what you’re looking for. Curious to watch replies here.

• gocron

  12 5,014 8.3 Go

  Easy and fluent Go cron scheduling. This is a fork from https://github.com/jasonlvhit/gocron

  

There is gocron which you would need to implement with a locking solution like in apscheduler perhaps with postgres or redis.

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• BeanstalkD

  14 6,490 0.0 C

  Beanstalk is a simple, fast work queue.

  

• aurora

  1 593 10.0 JavaScript

  Cross-platform beanstalkd queue server admin console. (by xuri)

• proposals

  60 64 4.2

  Temporal proposals (by temporalio)

  

Use Temporal open source project. It is much more than a cronjob service. But it supports managing periodic jobs natively.

• Airflow

  170 34,877 10.0 Python

  Apache Airflow - A platform to programmatically author, schedule, and monitor workflows

  

Airflow might also be a good option for you. Essentially DAGs of cronjobs. We like it a lot.

• Dkron

  3 4,187 8.2 Go

  Dkron - Distributed, fault tolerant job scheduling system https://dkron.io

  

Works great. Hope this helps. (For easiness GH here).

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Suggest a related project