Top 23 Go Concurrency Projects

• conc

  23 8,472 5.9 Go

  Better structured concurrency for go

  

Project mention: The Case of a Leaky Goroutine | news.ycombinator.com | 2024-03-25

It's a pity Go didn't have structured concurrency: https://vorpus.org/blog/notes-on-structured-concurrency-or-g...

There's a library for it: https://github.com/sourcegraph/conc

But this goes to one of the things I've been kind of banging on about languages, which is that if it's not in the language, or at least the standard library right at the beginning, sometimes it almost might as well not exist. Sometimes a new language can be valuable, even if it has no "new" language features, just to get a chance to reboot the standard library it has and push for patterns that older languages are theoretically capable of, but they just don't play well with any of the libraries in the language. Having it as a much-later 3rd party library just isn't good enough.

(In fact if I ever saw a new language start up and that was basically its pitch, I'd be very intrigued; it would show a lot of maturity in the language designer.)

• RxGo

  10 4,871 0.0 Go

  Reactive Extensions for the Go language.

  

Project mention: RxGo: Reactive Extensions for the Go Language | news.ycombinator.com | 2023-08-19

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• concurrent-map

  1 4,144 0.0 Go

  a thread-safe concurrent map for go

• golang-set

  6 3,944 4.7 Go

  A simple, battle-tested and generic set type for the Go language. Trusted by Docker, 1Password, Ethereum and Hashicorp.

• bild

  3 3,926 0.0 Go

  Image processing algorithms in pure Go

• hatchet

  16 3,304 9.7 Go

  A distributed, fault-tolerant task queue

  

Project mention: Ask HN: Who is hiring? (April 2024) | news.ycombinator.com | 2024-04-01

Hatchet (https://hatchet.run) | New York City | Full-time

We're hiring a founding engineer to help us with development on our open-source, distributed task queue: https://github.com/hatchet-dev/hatchet.

We recently launched on HN, you can check out our launch here: https://news.ycombinator.com/item?id=39643136. We're two second-time YC founders in this for the long haul and we are just wrapping up the YC W24 batch.

As a founding engineer, you'll be responsible for contributing across the entire codebase. We'll compensate accordingly and with high equity. It's currently just the two founders + a part-time contractor. We're all technical and contribute code.

Stack: Typescript/React, Go and PostgreSQL.

To apply, email alexander [at] hatchet [dot] run, and include the following:

1. Tell us about something impressive you've built.

2. Ask a question or write a comment about the state of the project. For example: a file that stood out to you in the codebase, a Github issue or discussion that piqued your interest, a general comment on distributed systems/task queues, or why our code is bad and how you could improve it.

• go-concurrency-patterns

  2 2,582 0.0 Go

  Concurrency patterns in Go

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• go-concurrency-guide

  4 2,362 0.0 Go

  Practical concurrency guide in Go, communication by channels, patterns

• go-web-framework-benchmark

  2 2,009 6.9 Go

  :zap: Go web framework benchmark

• workerpool

  4 1,253 0.0 Go

  Concurrency limiting goroutine pool (by gammazero)

Project mention: What 3rd-party libraries do you use often/all the time? | /r/golang | 2023-12-01

For worker pools of goroutines, this has always been "good enough" for me: https://github.com/gammazero/workerpool

• pond

  2 1,242 2.5 Go

  🔘 Minimalistic and High-performance goroutine worker pool written in Go

• go-wrk

  1 858 0.8 Go

  go-wrk - a HTTP benchmarking tool based in spirit on the excellent wrk tool (https://github.com/wg/wrk)

• slacker

  1 810 6.7 Go

  Slack Bot Framework

Project mention: Slack Bot Framework | /r/golang | 2023-06-29

We just released v2.0.0-alpha1 of https://github.com/shomali11/slacker

• ZenQ

  13 636 3.6 Go

  A thread-safe queue faster and more resource efficient than golang's native channels

• gowp

  1 492 3.8 Go

  golang worker pool , Concurrency limiting goroutine pool

• goconcurrentqueue

  1 365 0.0 Go

  Go concurrent-safe, goroutine-safe, thread-safe queue

• machine

  2 358 1.4 Go

  Machine is a zero dependency library for highly concurrent Go applications. It is inspired by errgroup.Group with extra bells & whistles (by autom8ter)

• semgroup

  1 289 2.6 Go

  Like errgroup/waitgroup, but only runs a maximum of tasks at any time.

• util

  0 286 0.0 Go

  A collection of useful utility functions (by shomali11)

• flowmatic

  2 254 7.3 Go

  Structured concurrency made easy

Project mention: Async rust – are we doing it all wrong? | news.ycombinator.com | 2023-07-19

I do that of course, and that's one of the easiest ways to use async Rust. In real projects you need much more however. F.ex. I had to code an example of how to add tasks to an already running pool of tasks and posted my findings here: https://github.com/dimitarvp/rust-async-examples/blob/main/e... (there's #2 as well with some more comments and a different approach).

The fact that I needed to make a GitHub repo and start making show-and-tell demos on how to do various things with async Rust to me is both a red flag and me being diligent BUT it should be more obvious. And promoted in docs.

Rust started suffering from "you got all the nuts and bolts in place, now build your own solution, son" syndrome which I grew to dislike. Too low-level. I wouldn't mind something akin to e.g. Golang's flowmatic library (check the first two examples at the top of the README): https://github.com/carlmjohnson/flowmatic

• ratelimiter

  3 238 0.0 Go

  A concurrent rate limiter library for Golang based on Sliding-Window rate limiter algorithm.

• theine-go

  5 221 6.9 Go

  high performance in-memory cache

Project mention: Otter, Fastest Go in-memory cache based on S3-FIFO algorithm | news.ycombinator.com | 2023-12-23

In fact, lock-free queues have several problems at once, which prompted me to give up on them almost immediately.

1. Yes, S3-FIFO can be implemented using lock-free queues, but the problem is that each write to a filled cache using this design will cause a large number of additional atomic operations not friendly to the processor's cache, while bp-wrapper on the contrary amortizes this load. And reading with frequency update on hot entries can have a bad effect on performance. In many ways this is exactly what the last posts in my discussion with Ben are about (not really about this, but the current problem with otter read speed is caused by a similar problem). https://github.com/Yiling-J/theine-go/issues/29#issuecomment...

2. But the main problem for me is not even that. Lock-free queues work fine as long as you only need to support Get and Set operations, but as soon as you want to add features to your cache, the complexity of the implementation starts to increase, and some features are very hard to add to such a structure. Also, improving the eviction policy is under a big question mark, because not only do you have to think about how to improve the eviction policy, but also how to avoid locks while doing so or how not to slow down the implementation with your improvements. BP-Wrapper has no such problems at all, allows you to use any eviction policy and focus on improving different parts of your cache independently of each other.

• zeropool

  1 213 4.7 Go

  Zero-allocation type-safe pool for Go

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Go Concurrency related posts

• The Case of a Leaky Goroutine

  3 projects | news.ycombinator.com | 25 Mar 2024

• Yazi: Fast terminal file manager based on async I/O

  7 projects | news.ycombinator.com | 15 Sep 2023

• RxGo: Reactive Extensions for the Go Language

  1 project | news.ycombinator.com | 19 Aug 2023

• Flowmatic: Structured concurrency made easy

  2 projects | /r/golang | 7 Jul 2023

• Show HN: Rendering Fractals with Goroutines

  1 project | news.ycombinator.com | 6 Jul 2023

• I’ve learned Go‘s concurrency model by rendering fractals

  1 project | /r/coolgithubprojects | 15 Jun 2023

• GO: How to Write a Worker Pool

  2 projects | dev.to | 15 Jun 2023
• A note from our sponsor - SaaSHub
  www.saashub.com | 22 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Go projects with our weekly report!