streaming-data

Open-source projects categorized as streaming-data

Top 23 streaming-data Open-Source Projects

  • awesome-bigdata

    A curated list of awesome big data frameworks, ressources and other awesomeness.

  • Project mention: Good coding groups for black women? | news.ycombinator.com | 2024-01-13
  • kafka-ui

    Open-Source Web UI for Apache Kafka Management

  • Project mention: FLaNK Stack Weekly 16 October 2023 | dev.to | 2023-10-17
  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • miller

    Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

  • Project mention: Qsv: Efficient CSV CLI Toolkit | news.ycombinator.com | 2023-12-22
  • Benthos

    Fancy stream processing made operationally mundane

  • Project mention: Benthos โ€“ Fancy stream processing made operationally mundane | news.ycombinator.com | 2024-05-15
  • materialize

    The data warehouse for operational workloads. (by MaterializeInc)

  • Project mention: The Notifier Pattern for Applications That Use Postgres | news.ycombinator.com | 2024-05-14

    Those updates are not retroactive. They apply on a go forward basis. Each day's changes become Apache 2.0 licensed on that day four years in the future.

    For example, v0.28 was released on October 18, 2022, and becomes Apache 2.0 licensed four years after that date (i.e., 2.5 years from today), on October 18, 2026.

    [0]: https://github.com/MaterializeInc/materialize/blob/76cb6647d...

  • river

    ๐ŸŒŠ Online machine learning in Python

  • Project mention: River: Online Machine Learning in Python | news.ycombinator.com | 2024-05-12
  • readyset

    Readyset is a MySQL and Postgres wire-compatible caching layer that sits in front of existing databases to speed up queries and horizontally scale read throughput. Under the hood, ReadySet caches the results of cached select statements and incrementally updates these results over time as the underlying data changes.

  • Project mention: Ask HN: How Can I Make My Front End React to Database Changes in Real-Time? | news.ycombinator.com | 2024-04-17

    - Some platforms like Supabase Realtime [3] and Firebase offer subscription models to database changes, but these solutions fall short when dealing with complex queries involving joins or group-bys.

    My vision is that the modern frontend to behave like a series of materialized views that dynamically update as the underlying data changes. Current state management libraries handle state trees well but don't seamlessly integrate with relational or graph-like database structures.

    The only thing I can think of is to implement it by myself, which sounds like a big PITA.

    Anything goes, Brainstorm with me. Is it causing you headaches as well? Are you familiar with an efficient solution? how are you all tackling it?

    [1] https://readyset.io/

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
  • smart_open

    Utils for streaming large files (S3, HDFS, gzip, bz2...)

  • fluvio

    Lean and mean distributed stream processing system written in rust and web assembly.

  • Project mention: Ask HN: WebSocket Relay? | news.ycombinator.com | 2024-02-27
  • Memgraph

    Open-source graph database, tuned for dynamic analytics environments. Easy to adopt, scale and own.

  • Project mention: Ask HN: Who is hiring? (March 2024) | news.ycombinator.com | 2024-03-01

    Memgraph | Staff C++ Database Engineer | REMOTE (Central/Western Europe, LatAm, or North America) https://memgraph.com/

    Memgraph is a Seed stage, open source graph database vendor. Graph DBs are a great solution for GenAI, logistics, cybersecurity and fintech so we are looking to grow aggressively this year.

    We're looking for a staff-level engineer to set technical direction, mentor junior team members, and solve some very difficult problems.

    Either DM me (the hiring manager) or apply here: https://join.com/companies/memgraph/10684850-staff-software-...

  • Pravega

    Pravega - Streaming as a new software defined storage primitive

  • go-streams

    A lightweight stream processing library for Go

  • bytewax

    Python Stream Processing

  • Project mention: Building a streaming SQL engine with Arrow and DataFusion | news.ycombinator.com | 2024-03-18
  • Streamz

    Real-time stream processing for python

  • zpl

    ๐Ÿ“ Pushing the boundaries of simplicity

  • OnlineStats.jl

    โšก Single-pass algorithms for statistics

  • scikit-multiflow

    A machine learning package for streaming data in Python. The other ancestor of River.

  • Project mention: ๐Ÿ”Underrated Open Source Projects You Should Know About ๐Ÿง  | dev.to | 2024-03-20

    River is actually the merger between creme and scikit-multiflow, another great example of open source collaboration and continuation.

  • hstream

    HStreamDB is an open-source, cloud-native streaming database for IoT and beyond. Modernize your data stack for real-time applications. (by hstreamdb)

  • Project mention: FLaNK Stack Weekly for 12 September 2023 | dev.to | 2023-09-12
  • awesome-kafka

    A list about Apache Kafka

  • streamdal

    Code-Native Data Pipelines

  • Project mention: Show HN: Streamdal โ€“ an open-source tail -f for your data | /r/hackernews | 2023-11-03
  • swim

    Full stack application platform for building stateful microservices, streaming APIs, and real-time UIs

  • kafka-ui

    Open-Source Web UI for managing Apache Kafka clusters (by kafbat)

  • Project mention: Show HN: Kafbat UI for Apache Kafka v1.0 is out | news.ycombinator.com | 2024-03-22
  • kafka-streams-in-action

    Source code for the Kafka Streams in Action Book

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

streaming-data related posts

  • Bento, the open source fork of the project formerly known as Benthos

    4 projects | news.ycombinator.com | 31 May 2024
  • Benthos โ€“ Fancy stream processing made operationally mundane

    1 project | news.ycombinator.com | 15 May 2024
  • Fancy stream processing made operationally mundane

    1 project | news.ycombinator.com | 6 Aug 2023
  • Benthos: Fancy stream processing made operationally mundane

    1 project | news.ycombinator.com | 15 Jul 2023
  • Need help on cleaning this data!!

    1 project | /r/datacleaning | 13 Jun 2023
  • Running weekly average

    1 project | /r/bash | 10 Jun 2023
  • johnkerl/miller: Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON

    1 project | /r/devel | 8 Jun 2023
  • A note from our sponsor - SaaSHub
    www.saashub.com | 31 May 2024
    SaaSHub helps you find the best software and product alternatives Learn more โ†’

Index

What are some of the best open-source streaming-data projects? This list will help you:

Project Stars
1 awesome-bigdata 12,861
2 kafka-ui 8,739
3 miller 8,614
4 Benthos 7,724
5 materialize 5,614
6 river 4,816
7 readyset 3,945
8 smart_open 3,102
9 fluvio 2,711
10 Memgraph 2,147
11 Pravega 1,976
12 go-streams 1,770
13 bytewax 1,265
14 Streamz 1,217
15 zpl 962
16 OnlineStats.jl 821
17 scikit-multiflow 747
18 hstream 693
19 awesome-kafka 566
20 streamdal 539
21 swim 473
22 kafka-ui 355
23 kafka-streams-in-action 259

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com