Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →
Top 23 Queue Open-Source Projects
-
Redisson
Redisson - Easy Valkey/Redis Java client and Real-Time Data Platform. Sync/Async/RxJava/Reactive API. Over 50 Redis based Java objects and services: Set, Multimap, SortedSet, Map, List, Queue, Deque, Semaphore, Lock, AtomicLong, Map Reduce, Bloom filter, Spring Cache, Tomcat, Scheduler, JCache API, Hibernate, RPC, local cache ...
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
Resque
Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
-
Hangfire
An easy way to perform background job processing in .NET and .NET Core applications. No Windows Service or separate process required
-
CAP
Distributed transaction solution in micro-service base on eventually consistency, also an eventbus with Outbox pattern
-
boltons
🔩 Like builtins, but boltons. 250+ constructs, recipes, and snippets which extend (and rely on nothing but) the Python standard library. Nothing like Michael Bolton.
-
algodeck
An Open-Source Collection of 200+ Flash Cards to Help You Preparing Your Algorithms & Data Structures Interview 💯
-
Jocko
Kafka implemented in Golang with built-in coordination (No ZK dep, single binary install, Cloud Native)
-
Coravel
Near-zero config .NET library that makes advanced application features like Task Scheduling, Caching, Queuing, Event Broadcasting, and more a breeze!
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Project mention: NSQ: Open-source realtime distributed messaging, billions of messages / day | news.ycombinator.com | 2024-01-09
Project mention: Prioritizing Concurrent Requests: Queuing system to handle distributed processes and messages with NodeJS and Bull | dev.to | 2024-03-15To implement the solution with a queue, I used a package called 'Bull' (https://github.com/OptimalBits/bull). It's a library that helps with distributed job control, providing some very useful solutions for this type of work, such as background job processing, queues with priorities (FIFO, LIFO, and others), among other features. 'Bull' uses Redis for queue storage, so if your application crashes for any reason, once it's back online, it will continue executing the processes that are in the queue. In our case, we'll use the FIFO (First in, first out) queue solution, meaning priority based on arrival order.
Hi there! I want to show off a little feature I made using hanami, htmx and a little bit of redis + sidekiq.
That's pretty cool. Reckon it would work with existing code that calls Redis over the wire for RQ?
https://python-rq.org
Project mention: 20 Top C# Frameworks and Libraries on GitHub for Building Powerful Applications | dev.to | 2024-05-236. Hangfire
Project mention: Boltons is a set of over 250 BSD-licensed, pure-Python utilities | news.ycombinator.com | 2023-12-11
Graphile worker has been great for me because it's a library that works with Postgres that allows me to queue jobs and execute them on the server without adding too many additional layers of complexity for being able to accomplish async tasks. (I'm aware of how popular bull is, but I don't want to add another data-store only for async tasks)
I've considered using Nextflow for bioinformatics pipelines but have yet to take the plunge. At work, I develop a proteomics pipeline that is composed of huey¹ tasks (Python library; simple alternative to Celery) which either use subprocess to call out to some external tool, or are just pure python. It runs in a worker container which is created by docker swarm, and all containers pull jobs from redis. For our scale, it works great. However, I don't have control over the resource utilization of individual steps, and in the past I've had issues with the pipeline blocking as a result of how I was chaining tasks together. I think something like Nextflow would remove these limitations, but one thing I think I would miss is the ability to debug individual pipeline steps locally with an interactive debugger. As far as I can tell, Nextflow has logging/tracing facilities but nothing quite like an interactive debugger. I'd be happy to be told I'm wrong, or even that I'm doing it wrong.
____
¹ https://github.com/coleifer/huey/
Queue related posts
-
Show HN: An SQS Alternative on Postgres
-
Hanami and HTMX - progress bar
-
Hangfire – Background Processing in .NET and .NET Core Applications
-
solid_queue alternatives - Sidekiq and good_job
3 projects | 21 Apr 2024 -
How to Use Flume in your Elixir Application
-
Redis Re-Implemented with SQLite
-
Prioritizing Concurrent Requests: Queuing system to handle distributed processes and messages with NodeJS and Bull
-
A note from our sponsor - InfluxDB
www.influxdata.com | 2 Jun 2024
Index
What are some of the best open-source Queue projects? This list will help you:
Project | Stars | |
---|---|---|
1 | nsq | 24,649 |
2 | Redisson | 22,827 |
3 | gods | 15,629 |
4 | bull | 15,153 |
5 | Sidekiq | 12,990 |
6 | rq | 9,572 |
7 | Resque | 9,392 |
8 | agenda | 9,279 |
9 | Hangfire | 9,115 |
10 | machinery | 7,350 |
11 | MassTransit | 6,612 |
12 | CAP | 6,493 |
13 | boltons | 6,430 |
14 | algodeck | 5,484 |
15 | The gist | 5,433 |
16 | huey | 4,936 |
17 | Jocko | 4,885 |
18 | Delayed::Job | 4,802 |
19 | PHP AMQP | 4,419 |
20 | horizon | 3,805 |
21 | Coravel | 3,644 |
22 | swift-collections | 3,571 |
23 | p-queue | 3,283 |
Sponsored