Top 23 Distributed System Open-Source Projects

• advanced-java

  1 74,573 6.3 Java

  😮 Core Interview Questions & Answers For Experienced Java(Backend) Developers | 互联网 Java 工程师进阶知识完全扫盲：涵盖高并发、分布式、高可用、微服务、海量数据处理等领域知识

  

• awesome-scalability

  7 53,537 6.2

  The Patterns of Scalable, Reliable, and Performant Large-Scale Systems

Project mention: Scalability – Patterns of Scalable and Reliable Large-Scale Systems | news.ycombinator.com | 2024-05-07

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• etcd

  62 46,505 9.9 Go

  Distributed reliable key-value store for the most critical data of a distributed system

  

Project mention: Chaos (fault) testing method for etcd and MongoDB | dev.to | 2024-05-19

so far, we have been able to do some fault simulation tests by using the fault points built in etcd. how to use these fault points can refer to the official integration test implementation of etcd-> etcd Robustness Testing. you can search for relevant codes by the name of the fault point.

• Dubbo

  1 40,093 9.7 Java

  The java implementation of Apache Dubbo. An RPC and microservice framework.

  

• system-design

  32 29,336 4.3

  Learn how to design systems at scale and prepare for system design interviews

Project mention: Learn how to design systems at scale and prepare for system design interviews | news.ycombinator.com | 2023-07-05

There are things in that doc that are completly wrong and ridiculous such as:

https://github.com/karanpratapsingh/system-design#streaming-...

• spacedrive

  31 29,029 9.9 TypeScript

  Spacedrive is an open source cross-platform file explorer, powered by a virtual distributed filesystem written in Rust.

  

Project mention: Interview with Mo Rajabi, co-founder and CEO of Noor | dev.to | 2024-04-18

In the video, Mo talked about a few packages like Cidre and StrOm, and we referred to SpaceDrive.

• xgboost

  11 25,636 9.6 C++

  Scalable, Portable and Distributed Gradient Boosting (GBDT, GBRT or GBM) Library, for Python, R, Java, Scala, C++ and more. Runs on single machine, Hadoop, Spark, Dask, Flink and DataFlow

  

Project mention: stackgbm VS xgboost - a user suggested alternative | libhunt.com/r/stackgbm | 2024-05-05

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

• nsq

  14 24,605 6.1 Go

  A realtime distributed messaging platform

  

Project mention: NSQ: Open-source realtime distributed messaging, billions of messages / day | news.ycombinator.com | 2024-01-09

• go-micro

  23 21,438 6.5 Go

  A Go microservices framework

  

• seaweedfs

  36 21,243 9.9 Go

  SeaweedFS is a fast distributed storage system for blobs, objects, files, and data lake, for billions of files! Blob store has O(1) disk seek, cloud tiering. Filer supports Cloud Drive, cross-DC active-active replication, Kubernetes, POSIX FUSE mount, S3 API, S3 Gateway, Hadoop, WebDAV, encryption, Erasure Coding.

  

Project mention: Ask HN: What distributed file system would you use in 2024? | news.ycombinator.com | 2024-05-10

• grpc-go

  30 19,970 9.6 Go

  The Go language implementation of gRPC. HTTP/2 based RPC

  

Project mention: Consistent Hashing: An Overview and Implementation in Golang | dev.to | 2024-05-07

grpc-go: go get -u google.golang.org/grpc

• rqlite

  112 14,948 9.9 Go

  The lightweight, distributed relational database built on SQLite.

  

Project mention: The lightweight, easy-to-use, distributed relational database built on SQLite | news.ycombinator.com | 2024-02-23

• NATS

  106 14,878 9.8 Go

  High-Performance server for NATS.io, the cloud and edge native messaging system.

  

Project mention: Implementing OTel Trace Context Propagation Through Message Brokers with Go | dev.to | 2024-04-04

Several message brokers, such as NATS and database queues, are not supported by OpenTelemetry (OTel) SDKs. This article will guide you on how to use context propagation explicitly with these message queues.

• Nomad

  95 14,468 9.9 Go

  Nomad is an easy-to-use, flexible, and performant workload orchestrator that can deploy a mix of microservice, batch, containerized, and non-containerized applications. Nomad is easy to operate and scale and has native Consul and Vault integrations.

  

Project mention: Ask HN: Are there any open source forks of nomad smd consul? | news.ycombinator.com | 2024-05-02

Doesn't look like it.

* https://github.com/hashicorp/nomad/forks

* https://github.com/search?q=nomad%20fork&type=repositories

* https://www.google.com/search?q=hashicorp+nomad+forks

There are products that do similar things of course.

• Akka

  33 12,936 9.4 Scala

  Build highly concurrent, distributed, and resilient message-driven applications on the JVM

  

Project mention: Modern Async Primitives on iOS, Android, and the Web | dev.to | 2023-12-06

Kotlin also has a construct for asynchronous collections/streams. Kotlin's version of AsyncSequence is called a Flow. Just as Swift's AsyncSequence builds upon prior experience with RxSwift and Combine, Kotlin's Flow APIs build upon earlier stream/collection APIs in the JVM ecosystem: Java's RxJava, Java8 Streams, Project Reactor, and Scala's Akka.

• micro

  42 12,029 8.7 Go

  A Go service development platform (by micro)

  

Project mention: Micro – A Go service development platform | news.ycombinator.com | 2024-04-24

• Apache ZooKeeper

  36 11,951 8.3 Java

  Apache ZooKeeper

  

Project mention: On Implementation of Distributed Protocols | dev.to | 2024-04-05

Apache ZooKeeper — a distributed coordination, synchronization, and configuration service (written in Java);

• conductor

  7 11,027 9.3 Java

  Conductor is an event driven orchestration platform (by conductor-oss)

  

Project mention: Emerging Tech Trends 2024: The Latest Developments in AI, API, and Automation | dev.to | 2024-05-17

Conductor is an open-source orchestration platform that automates complex or long-running processes, such as AI integration flows, microservice application flows, DevOps processes, transactional flows, and more. With Conductor, developers can build and update durable workflows without the complexities of managing system failures, dependencies, or scalability.

• awesome-distributed-systems

  6 10,232 3.0

  A curated list to learn about distributed systems

Project mention: Ask HN: How do I find technical papers (for distributed systems)? | news.ycombinator.com | 2023-07-03

Distributed systems is a very broad research topic nowadays, so it might make sense to check conferences covering a narrower topic, e.g. Cloud computing. Google Scholar might give some useful results for more specific search terms.

The reading lists for distributed systems university courses might also be interesting as well as this "awesome" link list on github:

https://github.com/theanalyst/awesome-distributed-systems

• NebulaGraph Database

  8 10,217 8.1 C++

  A distributed, fast open-source graph database featuring horizontal scalability and high availability (by vesoft-inc)

  

• temporal

  16 10,128 9.9 Go

  Temporal service

  

Project mention: Rethinking Serverless with Flame | news.ycombinator.com | 2023-12-06

I don't know if I agree with the argument regarding durability vs elastic execution. If I can get both (with a nice API/DX) via something like Temporal (https://github.com/temporalio/temporal), what's the drawback here?

• Orleans

  35 9,868 9.0 C#

  Cloud Native application framework for .NET

  

Project mention: .NET 8 – .NET Blog | news.ycombinator.com | 2023-11-14

We should chat more about Reminders v2 - I want to learn more about what you'd like it to provide. Can you comment on the issue (https://github.com/dotnet/orleans/issues/7573)? If you'd like to see some of what is in the works, please message me on Discord and we can chat more about it (https://aka.ms/orleans/discord)

• juicefs

  45 9,851 9.8 Go

  JuiceFS is a distributed POSIX file system built on top of Redis and S3.

  

Project mention: Data Sync in JuiceFS 1.2: Enhanced Selective Sync and Performance Optimizations | dev.to | 2024-05-17

In JuiceFS 1.2, we introduced several new features for juicefs sync. We also optimized performance for multiple scenarios to improve users' data synchronization efficiency when dealing with large directories and complex migrations.

• SaaSHub

  www.saashub.com featured

  SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

  

Distributed Systems related posts

• Show HN: Interactive Graph by LLM (GPT-4o)

  5 projects | news.ycombinator.com | 19 May 2024

• Why Kubernetes Was a Mistake for My SaaS Business (🤯)

  1 project | dev.to | 19 May 2024

• Emerging Tech Trends 2024: The Latest Developments in AI, API, and Automation

  1 project | dev.to | 17 May 2024

• AI Orchestration Meetup [April 2024] - Recap

  1 project | dev.to | 13 May 2024

• Consistent Hashing: An Overview and Implementation in Golang

  3 projects | dev.to | 7 May 2024

• Configure Renovate to update preview versions of NuGet packages

  2 projects | dev.to | 4 May 2024

• Micro – A Go service development platform

  1 project | news.ycombinator.com | 24 Apr 2024
• A note from our sponsor - SaaSHub
  www.saashub.com | 20 May 2024

  SaaSHub helps you find the best software and product alternatives Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com